-- Logs begin at Mon 2021-05-24 09:02:54 UTC, end at Mon 2021-05-24 10:25:03 UTC. -- May 24 09:51:46.373999 dv-cinder-791239-2-iscsi systemd[1]: Started Devstack devstack@q-svc.service. May 24 09:51:47.346864 dv-cinder-791239-2-iscsi neutron-server[109005]: Beware of the coming changes in the project packaging May 24 09:51:47.347471 dv-cinder-791239-2-iscsi neutron-server[109005]: https://github.com/svinota/pyroute2/discussions/786 May 24 09:51:48.813122 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.common.config [-] Logging enabled! May 24 09:51:48.813725 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev231 May 24 09:51:48.813811 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=109005) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 24 09:51:48.816954 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.common.config [-] Logging enabled! May 24 09:51:48.817231 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev231 May 24 09:51:48.817541 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=109005) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 24 09:51:48.818508 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 09:51:48.818742 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] Configuration options gathered from: {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 09:51:48.819095 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 09:51:48.819338 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 09:51:48.819564 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ================================================================================ {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 09:51:48.819889 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] allow_bulk = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.820184 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] allow_overlapping_ips = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.820480 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] api_extensions_path = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.820710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] api_paste_config = api-paste.ini {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.821003 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] api_workers = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.821323 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] auth_strategy = keystone {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.821608 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] backlog = 4096 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.821896 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] base_mac = fa:16:3e:00:00:00 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.822188 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] bind_host = 0.0.0.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.822477 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] bind_port = 9696 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.822753 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] client_socket_timeout = 900 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.823035 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] config_dir = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.823265 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.823610 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] config_source = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.823973 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] control_exchange = neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.824259 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] core_plugin = ml2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.824483 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] debug = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.824763 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] default_availability_zones = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.825226 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.825644 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] dhcp_agent_notification = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.825928 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] dhcp_lease_duration = 86400 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.826219 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] dns_domain = openstacklocal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.826499 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] enable_traditional_dhcp = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.826772 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] external_dns_driver = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.827049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] filter_validation = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.827334 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] global_physnet_mtu = 1500 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.827659 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] host = dv-cinder-791239-2-iscsi {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.827936 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] http_retries = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.828220 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] instance_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.828505 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.828793 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ipam_driver = internal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.829081 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ipv6_pd_enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.829361 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_config_append = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.829650 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.829923 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_dir = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.830195 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.830470 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_rotate_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.830759 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_rotate_interval_type = days {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.831042 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] log_rotation_type = none {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.831329 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.831554 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.831776 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.832062 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.832348 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.832625 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] max_dns_nameservers = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.832898 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] max_header_line = 16384 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.833202 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] max_logfile_count = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.833479 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] max_logfile_size_mb = 200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.833753 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] max_subnet_host_routes = 20 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.834021 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] network_link_prefix = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.834303 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] notify_nova_on_port_data_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.834585 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] notify_nova_on_port_status_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.834868 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] pagination_max_limit = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.835142 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] periodic_fuzzy_delay = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.835416 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] periodic_interval = 40 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.835685 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] publish_errors = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.835956 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rate_limit_burst = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.836239 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rate_limit_except_level = CRITICAL {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.836511 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rate_limit_interval = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.836789 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] retry_until_window = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.837112 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rpc_response_max_timeout = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.837402 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rpc_state_report_workers = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.837671 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] rpc_workers = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.837945 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] send_events_interval = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.838242 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.838527 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] setproctitle = on {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.838813 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] state_path = /opt/stack/data/neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.839124 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] syslog_log_facility = LOG_USER {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.839394 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] tcp_keepidle = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.839865 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.840203 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_eventlog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.840295 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_journal = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.840500 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_json = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.840775 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.841041 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_stderr = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.841344 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] use_syslog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.841617 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] vlan_transparent = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.841892 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] watch_log_file = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.842166 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] wsgi_default_pool_size = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.842439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] wsgi_keep_alive = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.842725 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:48.843021 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_concurrency.disable_process_locking = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.843318 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.843619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.connection_string = messaging:// {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.843904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.844198 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.es_doc_type = notification {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.844481 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.es_scroll_size = 10000 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.844771 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.es_scroll_time = 2m {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.845073 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.filter_error_trace = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.845370 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.hmac_keys = SECRET_KEY {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.845663 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.sentinel_service_name = mymaster {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.845957 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.socket_timeout = 0.1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.846237 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] profiler.trace_sqlalchemy = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.846548 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.enforce_new_defaults = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.846829 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.enforce_scope = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.847124 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.policy_default_rule = default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.847427 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.847722 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.848016 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.848292 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.848568 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.848843 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.849149 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.849437 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.849734 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.auth_type = password {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.850027 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.850302 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.850581 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.850872 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.851149 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.851423 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.851698 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.live_migration_events = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.851990 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.region_name = RegionOne {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.852270 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.852544 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] nova.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.852828 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.853138 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.853413 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.853685 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.853964 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.854253 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.854529 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.855346 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.855713 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.855992 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.856267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] placement.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.856553 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.856831 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.857138 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.857411 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.857690 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.857963 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.connect_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.858238 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.connect_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.858516 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.enable_notifications = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.858790 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.endpoint_override = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.859065 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.859337 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.interface = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.859637 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.859918 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.max_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.860193 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.min_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.860466 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.860738 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.service_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.861012 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.service_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.861312 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.861584 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.status_code_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.861855 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.status_code_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.862126 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.862399 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.valid_interfaces = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.862674 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ironic.version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.862960 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] service_providers.service_provider = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.863257 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.863536 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.863812 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.864091 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.864369 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.864647 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.864921 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.865227 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.865508 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.865783 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.866068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.866353 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.866639 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.866920 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.867197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.867490 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.867766 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.868047 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.868327 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.868605 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.868892 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.869198 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.869522 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.869880 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.870182 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.871046 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_version = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.871493 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_notifications.driver = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.872189 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_notifications.retry = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.872706 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.873000 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] oslo_messaging_notifications.transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:48.874170 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 09:51:48.874170 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo.service.wsgi [-] Loading app neutron from /etc/neutron/api-paste.ini {{(pid=109005) load_app /usr/local/lib/python3.8/dist-packages/oslo_service/wsgi.py:349}} May 24 09:51:49.226702 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=109005) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:51:49.226894 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" released by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.001s {{(pid=109005) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:51:49.227171 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=109005) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:51:49.227497 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [-] Loading core plugin: ml2 May 24 09:51:49.617026 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.617332 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.617602 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.618840 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.619110 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.619369 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.654698 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.654970 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.655235 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.668123 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.668397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.668662 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.698259 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.699063 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.699328 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.735684 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.735959 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.736226 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.967494 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.968120 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.968120 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.968290 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:49.968931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.097007 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.097353 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.097637 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.097913 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.098182 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.098491 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.098754 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.099009 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.099262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.099541 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.099798 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.100061 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.100332 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.100587 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.100852 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.101145 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.101403 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.101814 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.102067 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.102346 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.102692 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.102944 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.103193 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.103473 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.103721 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.103965 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.104347 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.104584 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.104845 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.105102 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.105338 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.105570 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:50.105927 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] May 24 09:51:50.108988 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1'] May 24 09:51:50.113251 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 24 09:51:50.117313 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(50, 99)])]) May 24 09:51:50.118914 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan'] May 24 09:51:50.119356 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan']) May 24 09:51:50.119690 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vlan'] May 24 09:51:50.120073 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security'] May 24 09:51:50.123113 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security'] May 24 09:51:50.123358 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security'] May 24 09:51:50.123701 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'hyperv'] May 24 09:51:50.151239 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitch [-] Open vSwitch agent's supported_vnic_types: ['normal', 'direct', 'smart-nic', 'vdpa'] May 24 09:51:50.152243 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > qos_plugin after_init 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.152539 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.qos.drivers.openvswitch.driver [-] Open vSwitch QoS driver registered {{(pid=109005) register /opt/stack/neutron/neutron/services/qos/drivers/openvswitch/driver.py:82}} May 24 09:51:50.152899 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > logging-plugin after_init 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.154260 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.logapi.drivers.openvswitch.driver [-] Open vSwitch logging driver registered {{(pid=109005) register /opt/stack/neutron/neutron/services/logapi/drivers/openvswitch/driver.py:69}} May 24 09:51:50.154543 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'hyperv'] May 24 09:51:50.154789 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'hyperv'] May 24 09:51:50.157966 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.158243 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:50.236231 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 24 09:51:50.236478 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 24 09:51:50.236748 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 24 09:51:50.237160 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 1000)] May 24 09:51:50.267302 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_db.sqlalchemy.engines [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109005) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 09:51:50.752556 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing driver for type 'gre' May 24 09:51:50.753109 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] gre ID ranges: [(1, 1000)] May 24 09:51:50.962791 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing driver for type 'local' May 24 09:51:50.963158 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing driver for type 'vlan' May 24 09:51:51.078403 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] VlanTypeDriver initialization complete May 24 09:51:51.079027 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing driver for type 'vxlan' May 24 09:51:51.079300 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] vxlan ID ranges: [(1, 1000)] May 24 09:51:51.300040 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing extension driver 'port_security' May 24 09:51:51.300379 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.extensions.port_security [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] PortSecurityExtensionDriver initialization complete May 24 09:51:51.300707 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing mechanism driver 'openvswitch' May 24 09:51:51.301030 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing mechanism driver 'hyperv' May 24 09:51:51.309717 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.plugins.ml2.plugin [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=109005) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:51:51.310460 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.310827 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.311174 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.311638 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.311987 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.312329 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.312708 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.313072 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.313415 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.313790 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.314136 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.314472 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.314956 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group_rule after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.315306 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group_rule after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.315643 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > security_group_rule after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.316012 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > address_group after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.316347 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > address_group after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.316700 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > address_group after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.317070 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] ML2 OVO RPC backend initialized. {{(pid=109005) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:160}} May 24 09:51:51.317776 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.318126 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.318536 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.318879 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > networks before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.319248 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.319588 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > ports before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.319920 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.320320 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnets before_response 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.320666 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.321198 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.321539 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.322190 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.322543 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.322879 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.323216 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.323579 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.323912 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > subnet after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.324506 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.plugin [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Modular L2 Plugin initialization complete May 24 09:51:51.324829 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading services supported by the core plugin {{(pid=109005) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:168}} May 24 09:51:51.325189 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.plugins.ml2.managers [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Got port-security extension from driver 'port_security' May 24 09:51:51.325648 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.extensions.vlantransparent [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Disabled vlantransparent extension. May 24 09:51:51.326134 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading service plugins: ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=109005) _load_service_plugins /opt/stack/neutron/neutron/manager.py:194}} May 24 09:51:51.326441 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: router May 24 09:51:51.364909 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.365325 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_create 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.365695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router before_create 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.366055 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.366423 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.366774 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_delete 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.367129 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > floatingip after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.367486 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_create 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.367828 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.368173 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.368514 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_gateway after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.368864 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.369290 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.369656 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.370008 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.370345 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: subnet after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.370688 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: subnetpool_address_scope after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.371036 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > port precommit_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.371378 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_create 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.371929 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.372278 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.372632 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router after_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.372970 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_delete 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.373353 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_create 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.373710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router_interface after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.374050 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_update 55549900 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.374415 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > network after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.375293 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Tracking information for resource: router configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:51.375623 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Tracking information for resource: floatingip configured {{(pid=109005) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 09:51:51.381134 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.db.l3_agentschedulers_db [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Skipping period L3 agent status check because automatic router rescheduling is disabled. May 24 09:51:51.381500 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.381859 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.382210 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.382655 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: port after_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.383457 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router before_create 55550100 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.383819 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_delete 55550100 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.384175 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_create 55550100 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.384526 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > router precommit_update 55550100 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.389083 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.provider_configuration [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service providers = [] {{(pid=109005) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}} May 24 09:51:51.389453 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.db.servicetype_db [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=109005) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}} May 24 09:51:51.392588 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.service_base [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded 'neutron.services.l3_router.service_providers.dvrha.DvrHaDriver' provider for service L3_ROUTER_NAT {{(pid=109005) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 09:51:51.392933 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.service_base [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded 'neutron.services.l3_router.service_providers.dvr.DvrDriver' provider for service L3_ROUTER_NAT {{(pid=109005) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 09:51:51.393303 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.service_base [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded 'neutron.services.l3_router.service_providers.ha.HaDriver' provider for service L3_ROUTER_NAT {{(pid=109005) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 09:51:51.394169 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.services.service_base [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded 'neutron.services.l3_router.service_providers.single_node.SingleNodeDriver' provider for service L3_ROUTER_NAT {{(pid=109005) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 09:51:51.394546 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.services.service_base [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Default provider is not specified for service type L3_ROUTER_NAT May 24 09:51:51.395097 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.395416 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: neutron.services.metering.metering_plugin.MeteringPlugin May 24 09:51:51.406864 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded METERING plugin. Description: METERING {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.407194 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: auto_allocate May 24 09:51:51.413767 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: network precommit_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.414139 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: network precommit_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.414690 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.414987 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: tag May 24 09:51:51.422414 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.422848 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: timestamp May 24 09:51:51.426879 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.427200 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: network_ip_availability May 24 09:51:51.432347 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.432713 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.433178 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.433951 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.434267 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: flavors May 24 09:51:51.451279 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.451610 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading Plugin: revisions May 24 09:51:51.455885 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=109005) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 09:51:51.456276 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Lock "manager" released by "neutron.manager.NeutronManager._create_instance" :: held 2.229s {{(pid=109005) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:51:51.457021 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] get_extension_paths = OrderedDict([('/opt/stack/neutron/neutron/extensions', 1)]) {{(pid=109005) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:635}} May 24 09:51:51.457377 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Initializing extension manager. May 24 09:51:51.457987 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: __init__.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.458387 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: __pycache__ {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.458691 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: _standard_attr_segment_lib.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.459020 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: address_group.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.459769 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.460332 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: address-group May 24 09:51:51.460651 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: address_scope.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.461369 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.461863 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: address-scope May 24 09:51:51.462183 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: admin_state_down_before_update.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.463086 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.463585 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: router-admin-state-down-before-update May 24 09:51:51.463906 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: agent.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.465228 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.465727 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: agent May 24 09:51:51.466045 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: agent_resources_synced.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.466928 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.467430 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: agent-resources-synced May 24 09:51:51.467747 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: allowedaddresspairs.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.469561 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.470068 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: allowed-address-pairs May 24 09:51:51.475446 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: auto_allocated_topology.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.476287 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.476796 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: auto-allocated-topology May 24 09:51:51.477142 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: availability_zone.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.477819 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.478316 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: availability_zone May 24 09:51:51.478632 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: availability_zone_filter.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.479506 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.480561 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: availability_zone_filter May 24 09:51:51.480873 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: data_plane_status.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.481786 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.482284 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension data-plane-status not supported by any of loaded plugins May 24 09:51:51.482598 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: default_subnetpools.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.483476 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.483971 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: default-subnetpools May 24 09:51:51.484286 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: dhcpagentscheduler.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.485107 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.485603 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: dhcp_agent_scheduler May 24 09:51:51.485919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: dns.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.486804 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.487373 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension dns-integration not supported by any of loaded plugins May 24 09:51:51.487690 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: dns_domain_ports.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.488575 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.489083 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension dns-domain-ports not supported by any of loaded plugins May 24 09:51:51.489404 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: dvr.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.490051 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.490541 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: dvr May 24 09:51:51.490856 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: empty_string_filtering.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.491741 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.492231 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: empty-string-filtering May 24 09:51:51.492546 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.493495 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.493984 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 24 09:51:51.494296 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.495172 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.495685 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 24 09:51:51.496003 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: external_net.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.496874 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.497408 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: external-net May 24 09:51:51.497722 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: extra_dhcp_opt.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.498638 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.499134 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: extra_dhcp_opt May 24 09:51:51.499549 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: extraroute.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.500426 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.500919 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: extraroute May 24 09:51:51.501264 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: extraroute_atomic.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.502280 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.502779 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: extraroute-atomic May 24 09:51:51.503092 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: filter_validation.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.503743 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.504228 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: filter-validation May 24 09:51:51.504538 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: fip_pf_description.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.505650 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.506141 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 24 09:51:51.506495 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: fip_port_details.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.507378 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.507875 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: fip-port-details May 24 09:51:51.508191 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: flavors.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.509431 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.509936 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: flavors May 24 09:51:51.510253 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.511962 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.512468 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 24 09:51:51.512785 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: floatingip_pools.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.513495 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.513990 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: floatingip-pools May 24 09:51:51.514308 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: ip_allocation.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.515204 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.515695 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension ip_allocation not supported by any of loaded plugins May 24 09:51:51.516052 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: ip_substring_port_filtering.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.522021 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.522523 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: ip-substring-filtering May 24 09:51:51.522840 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l2_adjacency.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.523821 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.524319 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension l2_adjacency not supported by any of loaded plugins May 24 09:51:51.524635 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.525709 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.526219 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: router May 24 09:51:51.526538 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3_conntrack_helper.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.529451 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.529974 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 24 09:51:51.530304 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3_ext_gw_mode.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.531252 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.531761 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: ext-gw-mode May 24 09:51:51.532091 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3_ext_ha_mode.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.533019 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.533544 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: l3-ha May 24 09:51:51.533863 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3_flavors.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.534862 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.535359 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: l3-flavors May 24 09:51:51.535680 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.536565 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.537075 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: l3-port-ip-change-not-allowed May 24 09:51:51.537405 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: l3agentscheduler.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.538460 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.538959 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: l3_agent_scheduler May 24 09:51:51.539280 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: logging.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.541220 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.541730 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension logging not supported by any of loaded plugins May 24 09:51:51.542054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: metering.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.543350 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.543925 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: metering May 24 09:51:51.544338 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.551171 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.551630 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: metering_source_and_destination_fields May 24 09:51:51.551958 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: multiprovidernet.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.553008 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.553552 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: multi-provider May 24 09:51:51.553869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: netmtu.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.554759 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.555260 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: net-mtu May 24 09:51:51.555579 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: netmtu_writable.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.556462 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.557095 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: net-mtu-writable May 24 09:51:51.557419 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: network_availability_zone.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.558114 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.558613 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: network_availability_zone May 24 09:51:51.558930 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: network_ip_availability.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.559991 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.560493 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: network-ip-availability May 24 09:51:51.560812 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: network_segment_range.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.561627 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.562121 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension network-segment-range not supported by any of loaded plugins May 24 09:51:51.562574 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: pagination.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.563515 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.563843 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: pagination May 24 09:51:51.564159 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: port_device_profile.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.565040 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.565564 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: port-device-profile May 24 09:51:51.565880 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: port_mac_address_regenerate.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.566763 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.567359 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: port-mac-address-regenerate May 24 09:51:51.567675 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: port_numa_affinity_policy.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.568600 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.569112 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: port-numa-affinity-policy May 24 09:51:51.569426 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: port_resource_request.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.570366 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.570850 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension port-resource-request not supported by any of loaded plugins May 24 09:51:51.571163 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: portbindings.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.572066 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.572558 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: binding May 24 09:51:51.572869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: portbindings_extended.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.574825 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.578959 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: binding-extended May 24 09:51:51.578959 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: portsecurity.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.579724 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.580251 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: port-security May 24 09:51:51.580575 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: project_id.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.581559 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.581887 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: project-id May 24 09:51:51.582208 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: providernet.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.583107 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.583616 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: provider May 24 09:51:51.583933 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.587582 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.588097 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos not supported by any of loaded plugins May 24 09:51:51.588419 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_bw_limit_direction.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.590222 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.590766 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 24 09:51:51.591169 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.592061 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.592562 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 24 09:51:51.592889 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_default.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.593812 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.594309 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-default not supported by any of loaded plugins May 24 09:51:51.594629 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_fip.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.595267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.595836 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-fip not supported by any of loaded plugins May 24 09:51:51.596157 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_gateway_ip.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.597113 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.597613 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-gateway-ip not supported by any of loaded plugins May 24 09:51:51.597930 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_port_network_policy.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.598808 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.599305 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-port-network-policy not supported by any of loaded plugins May 24 09:51:51.599621 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_rule_type_details.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.600509 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.601011 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-rule-type-details not supported by any of loaded plugins May 24 09:51:51.601355 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: qos_rules_alias.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.602372 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.602953 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension qos-rules-alias not supported by any of loaded plugins May 24 09:51:51.603270 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: quotasv2.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.604336 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per tenant" updated="2012-07-29T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.604659 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: quotas May 24 09:51:51.604995 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: quotasv2_detail.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.606911 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.607224 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: quota_details May 24 09:51:51.607553 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: rbac.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.608596 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.609134 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: rbac-policies May 24 09:51:51.609456 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: rbac_address_group.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.610372 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.610873 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: rbac-address-group May 24 09:51:51.611189 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: rbac_address_scope.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.612072 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.612573 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: rbac-address-scope May 24 09:51:51.612899 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: rbac_security_groups.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.613811 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.614309 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: rbac-security-groups May 24 09:51:51.614624 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: rbac_subnetpool.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.615508 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.616009 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: rbac-subnetpool May 24 09:51:51.616325 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: revisionifmatch.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.617236 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.617733 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: revision-if-match May 24 09:51:51.618049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: revisions.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.619125 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.619623 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: standard-attr-revisions May 24 09:51:51.619943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: router_availability_zone.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.620614 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.621144 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: router_availability_zone May 24 09:51:51.621462 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: routerservicetype.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.622348 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.622858 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension router-service-type not supported by any of loaded plugins May 24 09:51:51.623175 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.624060 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.624594 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: security-groups-normalized-cidr May 24 09:51:51.625023 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: security_groups_port_filtering.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.625943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.626445 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: port-security-groups-filtering May 24 09:51:51.626762 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: security_groups_remote_address_group.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.627652 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.628155 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: security-groups-remote-address-group May 24 09:51:51.628469 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: securitygroup.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.639961 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.640476 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: security-group May 24 09:51:51.640808 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: segment.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.641589 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.642086 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension segment not supported by any of loaded plugins May 24 09:51:51.642404 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.643313 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.643813 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 24 09:51:51.644131 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: servicetype.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.645235 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.645744 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: service-type May 24 09:51:51.646061 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: sorting.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.647007 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.647335 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: sorting May 24 09:51:51.647654 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: standard_attr_segment.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.648531 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.649037 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension standard-attr-segment not supported by any of loaded plugins May 24 09:51:51.649386 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: standardattrdescription.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.650010 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.650509 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: standard-attr-description May 24 09:51:51.650827 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: stateful_security_group.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.652271 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.652788 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: stateful-security-group May 24 09:51:51.653151 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: stdattrs_common.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.653721 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 24 09:51:51.654038 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.655006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.655507 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins May 24 09:51:51.655823 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnet_onboard.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.656894 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.657428 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: subnet_onboard May 24 09:51:51.657748 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnet_segmentid_writable.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.658676 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.659282 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 24 09:51:51.659611 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnet_service_types.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.660616 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.661239 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: subnet-service-types May 24 09:51:51.661560 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnetallocation.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.662685 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.663204 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: subnet_allocation May 24 09:51:51.663523 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.664174 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.664683 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: subnetpool-prefix-ops May 24 09:51:51.664995 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.665929 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.666429 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 24 09:51:51.666744 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: tagging.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.667829 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.668337 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: standard-attr-tag May 24 09:51:51.668659 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: timestamp.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.669857 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.670365 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loaded extension: standard-attr-timestamp May 24 09:51:51.670688 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: trunk.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.671689 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.672190 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension trunk not supported by any of loaded plugins May 24 09:51:51.672517 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: trunk_details.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.673445 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.674196 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension trunk-details not supported by any of loaded plugins May 24 09:51:51.674519 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: uplink_status_propagation.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.675412 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.675910 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension uplink-status-propagation not supported by any of loaded plugins May 24 09:51:51.676226 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Loading extension file: vlantransparent.py {{(pid=109005) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 09:51:51.676872 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=109005) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 09:51:51.677400 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extension vlan-transparent not supported by any of loaded plugins May 24 09:51:51.679873 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:network May 24 09:51:51.680597 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnet May 24 09:51:51.681401 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnetpool May 24 09:51:51.681998 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:port May 24 09:51:51.683274 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.683723 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service FLAVORS assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.684323 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.684736 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.685087 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:router May 24 09:51:51.685819 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:floatingip May 24 09:51:51.686499 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service METERING assigned prefix: /metering {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.687920 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.688356 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.688709 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=109005) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 09:51:51.689860 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of CountableResource for resource:rbac_policy May 24 09:51:51.690301 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:security_group May 24 09:51:51.690952 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:security_group_rule May 24 09:51:51.691675 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service CORE assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.692007 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnetpool May 24 09:51:51.692342 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] subnetpool is already registered May 24 09:51:51.692936 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service CORE assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.693301 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnetpool May 24 09:51:51.693670 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] subnetpool is already registered May 24 09:51:51.696141 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.696566 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service FLAVORS assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.697180 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.697593 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.697924 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:router May 24 09:51:51.698246 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] router is already registered May 24 09:51:51.699393 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:floatingip May 24 09:51:51.699783 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] floatingip is already registered May 24 09:51:51.700397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service METERING assigned prefix: /metering {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.701078 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of CountableResource for resource:rbac_policy May 24 09:51:51.701398 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] rbac_policy is already registered May 24 09:51:51.701928 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:security_group May 24 09:51:51.702260 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] security_group is already registered May 24 09:51:51.702838 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:security_group_rule May 24 09:51:51.703162 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] security_group_rule is already registered May 24 09:51:51.703816 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service CORE assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.704144 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnetpool May 24 09:51:51.704461 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] subnetpool is already registered May 24 09:51:51.705069 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.v2.resource_helper [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Service CORE assigned prefix: {{(pid=109005) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 09:51:51.705433 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Creating instance of TrackedResource for resource:subnetpool May 24 09:51:51.705752 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING neutron.quota.resource_registry [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] subnetpool is already registered May 24 09:51:51.706385 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: extensions {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.707846 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: address-groups {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.709481 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: address-scopes {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.710715 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: agents {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.711874 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: auto-allocated-topology {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.713177 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: availability_zones {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.715029 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: dhcp-networks {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.716460 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: dhcp-agents {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.718002 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: routers {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.719794 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: flavors {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.720964 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: service_profiles {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.722203 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: next_providers {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.723719 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: service_profiles {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.725204 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: floatingip-pools {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.726539 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: routers {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.728363 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: floatingips {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.729571 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: l3-routers {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.730976 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: l3-agents {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.732410 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: metering-labels {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.733792 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: metering-label-rules {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.735071 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: network-ip-availabilities {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.736324 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: bindings {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.738022 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: quotas {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.739520 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: quotas {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.740936 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: rbac-policies {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.743217 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: security-groups {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.744455 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: security-group-rules {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.745824 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: service-providers {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.747082 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: subnetpools {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.860103 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: subnetpools {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.861521 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.863239 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.865093 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.866879 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.868686 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.870606 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.872310 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.874252 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.876027 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.877915 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron.api.extensions [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Extended resource: tags {{(pid=109005) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 09:51:51.883681 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING keystonemiddleware.auth_token [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 09:51:51.889236 dv-cinder-791239-2-iscsi neutron-server[109005]: WARNING keystonemiddleware.auth_token [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 09:51:51.897349 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Acquired lock "singleton_lock" {{(pid=109005) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:266}} May 24 09:51:51.897825 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_concurrency.lockutils [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Releasing lock "singleton_lock" {{(pid=109005) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:282}} May 24 09:51:51.898445 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Starting 2 workers May 24 09:51:51.903756 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Started child 109112 {{(pid=109005) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 09:51:51.908095 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Started child 109113 {{(pid=109005) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 09:51:51.908991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.909824 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO neutron.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Neutron service started, listening on 0.0.0.0:9696 May 24 09:51:51.910324 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Notify callbacks [] for process, before_spawn {{(pid=109005) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.910565 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [-] (109112) wsgi starting up on http://0.0.0.0:9696 May 24 09:51:51.912796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.913332 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Starting 1 workers May 24 09:51:51.914166 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [-] (109113) wsgi starting up on http://0.0.0.0:9696 May 24 09:51:51.918049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Started child 109114 {{(pid=109005) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 09:51:51.918563 dv-cinder-791239-2-iscsi neutron-server[109005]: INFO oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Starting 1 workers May 24 09:51:51.921769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] calling RpcWorker wait() {{(pid=109114) _wait /opt/stack/neutron/neutron/service.py:128}} May 24 09:51:51.922325 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] returning from RpcWorker wait() {{(pid=109114) _wait /opt/stack/neutron/neutron/service.py:135}} May 24 09:51:51.922964 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Started child 109115 {{(pid=109005) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 09:51:51.923645 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.923754 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG neutron_lib.callbacks.manager [None req-668a19ef-67c2-46da-8295-812dccf05f9b None None] Notify callbacks [] for process, after_spawn {{(pid=109005) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.924435 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [-] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:51:51.925915 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=109005) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 24 09:51:51.926126 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 09:51:51.926307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 09:51:51.926525 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 09:51:51.926858 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 09:51:51.927020 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 09:51:51.927520 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.927898 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.928122 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:51:51.928588 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.928588 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.928727 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.929253 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.929540 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.929778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.930070 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.930329 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.930503 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.930669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.930911 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.931227 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.931511 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.931776 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.932033 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.932278 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.932448 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.932707 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_source = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.933083 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.933426 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.933704 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] debug = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.933967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.934474 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.934702 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.934941 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.935187 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.935448 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.935694 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.935976 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.936224 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.936458 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.936700 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.936951 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.937267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.937483 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.937724 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.937993 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.938209 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.938708 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] host = dv-cinder-791239-2-iscsi {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.938926 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.939141 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.939388 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.939663 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.939923 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.940171 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.940419 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.940662 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.940907 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.941160 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.941422 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.941653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.941906 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_dir = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.942142 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.942304 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_options = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.942525 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.942796 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.943053 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.943241 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.943409 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.943596 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.943844 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.944011 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.944257 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.944488 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.944700 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.944929 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.945184 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.945423 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.945679 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.945905 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.946118 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.946348 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.946619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.946845 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.947073 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.947298 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.947513 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.947747 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.947962 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.948181 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.948398 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.949187 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.949401 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.949661 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.949874 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.950134 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.950367 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.950615 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.950864 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.951094 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.951309 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.951599 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.951830 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.952076 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.952342 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.952557 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.953897 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:51:51.954059 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.954232 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.954479 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_journal = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.954720 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_json = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.956018 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.956018 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.956521 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.956663 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.956818 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.957000 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.957283 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.957539 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:51.957974 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.958274 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.958595 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.958841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.959088 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.959353 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.959611 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.959858 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.960113 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.960369 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.960665 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.960913 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.961216 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.961465 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.961735 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.962070 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.962323 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.962619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.962855 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.963094 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.963317 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.963547 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.963886 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.964205 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.965004 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.67/identity {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.967904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968397 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968751 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.968751 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.969018 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.969231 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.969452 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.969690 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.969927 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.970145 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.970374 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.970627 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.970869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.971117 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.971338 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.971554 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.971967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.972395 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.972561 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.972712 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.972853 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.973151 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.973319 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.973540 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.973800 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.974025 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.974247 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.974466 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.974688 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.974905 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.975128 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.975355 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.975593 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.975840 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.976068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.976301 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.976496 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.976712 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.976931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.977175 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.977414 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.977921 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.977921 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.978166 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.978250 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.978474 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.978710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.979003 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.979311 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.979564 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.979844 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.980088 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.980343 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.980567 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.980812 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.981094 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.981374 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.981602 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.981859 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.982120 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.982390 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.982642 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.982901 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.983158 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.983629 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.983737 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.983891 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.984142 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.984366 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.984642 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.984964 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.985250 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.985513 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.985813 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.986090 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.986365 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.986656 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.986880 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.987177 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.987477 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.987783 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.988030 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.988292 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.988750 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.988750 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.988992 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.989283 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.989511 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.989743 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.989958 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.990236 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.990470 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.990920 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.991625 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.992030 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.992174 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.993110 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.993110 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.993110 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.993110 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.994490 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.994490 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.994490 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.994490 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.995002 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.995002 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.995248 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.67/identity {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.995248 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.995993 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.998432 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999262 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999909 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:51.999909 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.000056 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.000144 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.000344 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.000561 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.001015 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.001256 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.001545 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.001801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.002038 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.002267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.002533 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.002794 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.003010 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.003943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.003943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.003943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.003943 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.004392 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.004392 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.004550 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.005095 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.005095 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.005355 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.005493 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.005776 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.006027 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.006258 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.006496 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.006715 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.006951 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.007223 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.007447 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.007661 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.007899 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.008135 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.008407 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.008667 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.008909 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.009220 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.009492 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.009711 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.009930 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.010149 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.010456 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.010702 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.010950 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.011731 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.011994 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.012224 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.012436 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.012951 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.012951 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.013213 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.013410 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.013502 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.015778 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016684 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016766 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.016969 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.017282 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.017514 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.017759 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.018003 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.018322 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.018576 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.018845 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.019065 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.019282 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.019505 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.019761 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.020077 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.020383 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.020623 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.020864 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.021176 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.021461 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:50:99'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.021722 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.021938 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.022231 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.022563 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.022830 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.023071 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.023315 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.023590 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.023821 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.024049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.024267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.024519 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.024749 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.024978 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.025229 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.025229 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.025742 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.025900 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.026100 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.026344 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.026610 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.026862 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.027274 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.027274 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 09:51:52.027793 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=109005) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 24 09:51:52.027952 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 09:51:52.028104 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 09:51:52.028299 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 09:51:52.028468 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 09:51:52.028626 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 09:51:52.028816 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.028971 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029156 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029299 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029438 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029613 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029772 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.029918 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030095 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030233 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030415 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030549 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030728 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030868 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.030995 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031112 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031268 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031428 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031592 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031751 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] config_source = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.031909 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.032067 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.032622 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] debug = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.032813 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.032968 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.033176 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.033337 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.033735 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.033735 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.034017 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.034017 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.034213 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.034213 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.037980 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.038194 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.038370 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.038556 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.038743 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.038931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.039101 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.039340 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] host = dv-cinder-791239-2-iscsi {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.039548 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.039721 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.040949 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041136 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041283 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041435 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041608 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041777 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.041942 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.042104 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.042291 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.043129 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.043286 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_dir = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.043496 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.043725 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_options = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.043725 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044374 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044374 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044714 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.044714 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045113 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045113 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045113 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045586 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045586 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045586 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045959 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.045959 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.047255 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.047419 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.047618 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.047758 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.048054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.048217 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.048541 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.048702 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.049299 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.049460 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.049625 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.049804 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.049947 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050091 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050246 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050407 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050567 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050725 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.050965 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.051159 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.051350 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.051530 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.051711 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.051885 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052114 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052287 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052377 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49310) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:51:52.052460 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052618 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052824 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.052989 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_journal = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053172 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_json = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053335 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053497 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053644 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053809 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.053972 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.054127 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.054267 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.054435 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:52.054621 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.054788 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.054964 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055130 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055282 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055445 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055596 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055761 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.055924 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056087 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056261 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056410 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056589 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056739 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.056904 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.057077 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.057305 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.057475 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.057724 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.058478 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.058478 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.058821 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.058821 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.67/identity {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.059919 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.060869 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.061669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.061669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.061669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.061669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.061669 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.062307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.062307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.062307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.062307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.062307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063054 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.063710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064234 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064234 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064503 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064503 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064764 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.064764 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065558 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065697 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [-] 10.222.0.67 "GET / HTTP/1.1" status: 200 len: 253 time: 0.0096502 May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.065841 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.066931 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.068967 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.070439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.070439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.070439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.070439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.070439 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072006 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072345 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072345 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072345 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072573 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072649 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.072830 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.073145 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.073145 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.073315 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.073465 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.074201 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.074369 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.074534 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.074700 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.074863 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.075077 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076406 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076937 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076937 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076937 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.076937 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077229 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077310 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077580 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077580 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077736 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.077898 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.078087 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.078087 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.078087 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.080801 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.67/identity {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.082695 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.084197 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.086300 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087108 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.087653 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.088619 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.089908 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.090797 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.091560 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.091560 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.091560 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.091560 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.091937 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:51:52.092068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.092068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.092068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.092068 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.092740 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.092923 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093159 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093308 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093459 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093596 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093733 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.093882 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094049 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094213 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094375 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094548 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094710 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.094873 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095035 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095198 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095371 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095526 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095683 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.095845 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096007 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096179 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096343 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096506 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096744 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.096898 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097044 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097206 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097343 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097479 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097625 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097774 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.097989 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098135 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098273 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098410 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098575 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098745 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.098897 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099069 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099240 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:50:99'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099410 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099574 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099747 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.099922 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100160 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100307 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100459 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100598 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100734 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.100870 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101007 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101214 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101352 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101489 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101625 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101773 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.101923 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102060 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102218 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102381 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102544 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102707 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.102870 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:52.103012 dv-cinder-791239-2-iscsi neutron-server[109005]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=109005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 09:52:03.296416 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-04a854dc-9ea2-4a20-a16e-83cf8af0d7e1 None None] get_active_networks_info from dv-cinder-791239-2-iscsi {{(pid=109114) get_active_networks_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:158}} May 24 09:52:03.365597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-04a854dc-9ea2-4a20-a16e-83cf8af0d7e1 None None] No non-hosted networks {{(pid=109114) auto_schedule_networks /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:60}} May 24 09:52:03.379115 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.agentschedulers_db [None req-04a854dc-9ea2-4a20-a16e-83cf8af0d7e1 None None] DHCP Agent not found on host dv-cinder-791239-2-iscsi {{(pid=109114) list_active_networks_on_active_dhcp_agent /opt/stack/neutron/neutron/db/agentschedulers_db.py:455}} May 24 09:52:03.529136 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-959c4ab6-7e05-4cdf-9bdf-7a55fca2bede None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:05.819174 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:05.825309 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.826957 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.826957 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.828605 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.829615 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.830536 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.831103 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.831666 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.832197 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.832784 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.833377 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.833910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:05.834569 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServersRpcApi method report_agent_resource_versions called with arguments (, 'Open vSwitch agent', 'dv-cinder-791239-2-iscsi', {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:05.846046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServerRpcCallback method report_agent_resource_versions called with arguments (,) {'agent_type': 'Open vSwitch agent', 'agent_host': 'dv-cinder-791239-2-iscsi', 'version_map': {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:05.894982 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1df27ab0-69d2-4827-bbb1-77c849585b29 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:05.959840 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.db.api [None req-04397a70-2c36-4d60-b6f4-fa78b641442d None None] Retry wrapper got retriable exception: Failed to create a duplicate Agent: for attribute(s) ['agents.uniq_agents0agent_type0host'] with value(s) L3 agent-dv-cinder-791239-2-iscsi {{(pid=109114) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 24 09:52:05.960695 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_db.api [None req-04397a70-2c36-4d60-b6f4-fa78b641442d None None] Performing DB retry for function neutron.db.agents_db.AgentDbMixin.create_or_update_agent {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 24 09:52:06.539949 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-04397a70-2c36-4d60-b6f4-fa78b641442d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:07.175511 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:08.233124 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.db.l3_hamode_db [None req-6750694b-2fab-41b5-bd16-7f798fd5711d None None] Number of active agents lower than max_l3_agents_per_router. L3 agents available: 1 May 24 09:52:08.320448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6750694b-2fab-41b5-bd16-7f798fd5711d None None] Routers set() bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 09:52:08.320865 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6750694b-2fab-41b5-bd16-7f798fd5711d None None] Router IDs set() for agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:464}} May 24 09:52:08.678230 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:09.947529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49410) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:10.533997 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING oslo_policy.policy [None req-95b77759-8196-4670-9eeb-ed44c009a293 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:10.534586 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING oslo_policy.policy [None req-95b77759-8196-4670-9eeb-ed44c009a293 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:10.751801 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [None req-95b77759-8196-4670-9eeb-ed44c009a293 admin admin] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:52:10.752889 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-95b77759-8196-4670-9eeb-ed44c009a293 admin admin] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8044567 May 24 09:52:13.135692 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49424) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:13.742652 dv-cinder-791239-2-iscsi neutron-server[109112]: WARNING oslo_policy.policy [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:13.743174 dv-cinder-791239-2-iscsi neutron-server[109112]: WARNING oslo_policy.policy [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:13.956516 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Request body: {'subnetpool': {'prefixes': ['172.20.0.0/16'], 'name': 'shared-default-subnetpool-v4', 'default_prefixlen': 26, 'shared': True, 'is_default': True}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:14.058876 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.policy [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=109112) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 24 09:52:14.104617 dv-cinder-791239-2-iscsi neutron-server[109112]: WARNING oslo_policy.policy [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:14.105171 dv-cinder-791239-2-iscsi neutron-server[109112]: WARNING oslo_policy.policy [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:14.113923 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.quota [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Loaded quota_driver: . May 24 09:52:14.129485 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:14.255771 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Notify callbacks ['neutron.objects.securitygroup.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075695368', 'neutron.objects.address_group.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075696955', 'neutron.objects.network.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075702273', 'neutron.objects.qos.policy.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075680701', 'neutron.objects.subnetpool.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075686448', 'neutron.objects.address_scope.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8762075860205', 'neutron.plugins.ml2.plugin.External_net_db_mixin._process_ext_policy_create-3553214', 'neutron.plugins.ml2.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-547900', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-3681387', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-3736064'] for rbac-policy, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:14.450258 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] Notify callbacks [] for subnetpool, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:14.451600 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3a885d53-a6c5-4059-9546-12b570bc3c38 admin admin] 10.222.0.67 "POST /v2.0/subnetpools HTTP/1.1" status: 201 len: 739 time: 1.3133051 May 24 09:52:17.111412 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49442) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:17.487934 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Request body: {'network': {'name': 'private', 'admin_state_up': True, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:17.493528 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:17.498454 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Usage tracker for resource:network and tenant:33a491aa066c4be7bc85d06159d12d9a is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:17.504008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:17.511732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:17.620526 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Made reservation on behalf of 33a491aa066c4be7bc85d06159d12d9a for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:52:17.623420 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:17.641318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:17.656352 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:17.660314 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Usage tracker for resource:security_group_rule and tenant:33a491aa066c4be7bc85d06159d12d9a is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:17.666099 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:17.673108 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:17.768471 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:17.883541 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:17.888512 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:17.907320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 70} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 09:52:17.932804 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8762070920174', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:17.933439 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Unsubscribe: segment precommit_create {{(pid=109112) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 24 09:52:17.933892 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Added segment 2051bcb0-6c86-4013-9786-b1de425619cb of type vlan for network ab4a7426-e455-41e3-a994-4e61d59242c8 May 24 09:52:17.981062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=15374580-a924-4141-9492-8e799ee702fb,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:18.001433 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.001786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003814 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.003919 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.004221 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.004523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.004828 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Tracked resource versions report: May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: pushed versions: May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: { 'AddressGroup': {'1.2'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'ConntrackHelper': {'1.0'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Log': {'1.0'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Network': {'1.1'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Port': {'1.7'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'PortForwarding': {'1.2'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'QosPolicy': {'1.8'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroup': {'1.4'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroupRule': {'1.2'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SubPort': {'1.0'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Subnet': {'1.1'}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Trunk': {'1.1'}} May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: consumer versions: May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'ConntrackHelper': '1.0', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Log': '1.0', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Network': '1.1', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Port': '1.7', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'PortForwarding': '1.2', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'QosPolicy': '1.8', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroup': '1.4', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroupRule': '1.2', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SubPort': '1.0', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Subnet': '1.1', May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Trunk': '1.1'}} May 24 09:52:18.005884 dv-cinder-791239-2-iscsi neutron-server[109112]: {{(pid=109112) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 09:52:18.054422 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:18.062244 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.174s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:18.174758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:18.176289 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:18.176921 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:18.182029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:18.284558 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Pushing event updated for resources: {'Network': ['ID=ab4a7426-e455-41e3-a994-4e61d59242c8,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:18.288030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:18.296138 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network ab4a7426-e455-41e3-a994-4e61d59242c8 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 09:52:18.347407 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 848 time: 1.2347710 May 24 09:52:20.978622 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49458) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:21.550467 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.policy [None req-93ec6712-9659-443b-b063-cac28dafe2e4 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=109113) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 24 09:52:21.596834 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING oslo_policy.policy [None req-93ec6712-9659-443b-b063-cac28dafe2e4 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:21.597235 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING oslo_policy.policy [None req-93ec6712-9659-443b-b063-cac28dafe2e4 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 09:52:21.606844 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-93ec6712-9659-443b-b063-cac28dafe2e4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:21.608519 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-93ec6712-9659-443b-b063-cac28dafe2e4 admin admin] 10.222.0.67 "GET /v2.0/networks/ab4a7426-e455-41e3-a994-4e61d59242c8 HTTP/1.1" status: 200 len: 824 time: 0.6289976 May 24 09:52:21.639420 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-57fef020-e7cf-4160-a391-da4e94a68f11 admin admin] 10.222.0.67 "GET /v2.0/subnetpools/5f5518c3-256f-4498-8245-e4e8406cf4e1 HTTP/1.1" status: 200 len: 734 time: 0.0255845 May 24 09:52:21.648162 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Request body: {'subnet': {'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'name': 'private-subnet', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'ip_version': 4, 'gateway_ip': '172.20.0.1'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:21.649687 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.quota [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Loaded quota_driver: . May 24 09:52:21.655766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:21.661300 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Usage tracker for resource:subnet and tenant:33a491aa066c4be7bc85d06159d12d9a is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:21.664554 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:21.671948 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:21.739046 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Made reservation on behalf of 33a491aa066c4be7bc85d06159d12d9a for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:52:21.741805 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:22.328879 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:22.334693 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:22.414726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Checking segment: {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:52:22.415082 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Network hostable DHCP agents. Network: ab4a7426-e455-41e3-a994-4e61d59242c8, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 09:52:22.429719 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Pushing event updated for resources: {'Subnet': ['ID=9e3a394e-579d-4b2f-a775-1e06acc3e42c,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:22.441334 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.441692 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.442561 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.442561 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.442749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.443039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.443356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.443682 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.444014 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.444337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.444630 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.444933 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Tracked resource versions report: May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: pushed versions: May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: { 'AddressGroup': {'1.2'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'ConntrackHelper': {'1.0'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Log': {'1.0'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Network': {'1.1'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Port': {'1.7'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'PortForwarding': {'1.2'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'QosPolicy': {'1.8'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroup': {'1.4'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroupRule': {'1.2'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SubPort': {'1.0'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Subnet': {'1.1'}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Trunk': {'1.1'}} May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: consumer versions: May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'ConntrackHelper': '1.0', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Log': '1.0', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Network': '1.1', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Port': '1.7', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'PortForwarding': '1.2', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'QosPolicy': '1.8', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroup': '1.4', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroupRule': '1.2', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SubPort': '1.0', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Subnet': '1.1', May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Trunk': '1.1'}} May 24 09:52:22.445994 dv-cinder-791239-2-iscsi neutron-server[109113]: {{(pid=109113) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 09:52:22.474116 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:22.540369 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Network ab4a7426-e455-41e3-a994-4e61d59242c8 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 09:52:22.655028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, ab4a7426-e455-41e3-a994-4e61d59242c8) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:22.660957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:22.661989 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:22.667346 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-de1d6f5b-0f82-4aea-8b22-97acb5cd4fd1 None None] Network ab4a7426-e455-41e3-a994-4e61d59242c8 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:52:22.752445 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 1.1077511 May 24 09:52:22.980850 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-ab4a7426-e455-41e3-a994-4e61d59242c8', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 09:52:22.982090 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:23.175120 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Allocated IP 172.20.0.2 (ab4a7426-e455-41e3-a994-4e61d59242c8/9e3a394e-579d-4b2f-a775-1e06acc3e42c/4fa48412-4c97-42b7-b7d6-55e36371a598) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 09:52:23.407420 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:23.420052 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Transition to ACTIVE for port object 4fa48412-4c97-42b7-b7d6-55e36371a598 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:52:23.440714 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Transition to ACTIVE for port object 4fa48412-4c97-42b7-b7d6-55e36371a598 will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:52:23.625829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:23.630950 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:23.690449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Network ab4a7426-e455-41e3-a994-4e61d59242c8 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 09:52:23.693445 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:23.698944 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 09:52:23.705606 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Pushing event updated for resources: {'Port': ['ID=4fa48412-4c97-42b7-b7d6-55e36371a598,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:23.743110 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.743462 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.743796 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.744120 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.744438 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.744758 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.745106 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.745402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.745728 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.746569 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.746876 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.747181 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Tracked resource versions report: May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: pushed versions: May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: { 'AddressGroup': {'1.2'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'ConntrackHelper': {'1.0'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Log': {'1.0'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Network': {'1.1'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Port': {'1.7'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'PortForwarding': {'1.2'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'QosPolicy': {'1.8'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroup': {'1.4'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroupRule': {'1.2'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SubPort': {'1.0'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Subnet': {'1.1'}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Trunk': {'1.1'}} May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: consumer versions: May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'ConntrackHelper': '1.0', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Log': '1.0', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Network': '1.1', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Port': '1.7', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'PortForwarding': '1.2', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'QosPolicy': '1.8', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroup': '1.4', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroupRule': '1.2', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SubPort': '1.0', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Subnet': '1.1', May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Trunk': '1.1'}} May 24 09:52:23.748247 dv-cinder-791239-2-iscsi neutron-server[109114]: {{(pid=109114) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 09:52:23.778720 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:23.803771 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Attempting to bind port 4fa48412-4c97-42b7-b7d6-55e36371a598 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 09:52:23.804157 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Attempting to bind port 4fa48412-4c97-42b7-b7d6-55e36371a598 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 09:52:23.804558 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Attempting to bind port 4fa48412-4c97-42b7-b7d6-55e36371a598 on network ab4a7426-e455-41e3-a994-4e61d59242c8 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 09:52:23.810361 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 9, 52, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 09:52:23.810771 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Checking segment: {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:52:23.811229 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:23.811614 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Bound using segment: {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 09:52:23.812106 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Bound port: 4fa48412-4c97-42b7-b7d6-55e36371a598, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 09:52:23.812467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:23.900351 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] For port 4fa48412-4c97-42b7-b7d6-55e36371a598, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 09:52:23.904521 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] For port 4fa48412-4c97-42b7-b7d6-55e36371a598, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4fa48412-4c97-42b7-b7d6-55e36371a598,segment=NetworkSegment(2051bcb0-6c86-4013-9786-b1de425619cb),segment_id=2051bcb0-6c86-4013-9786-b1de425619cb)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 09:52:24.019520 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:24.023537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:24.106597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Pushing event updated for resources: {'Port': ['ID=4fa48412-4c97-42b7-b7d6-55e36371a598,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:24.109627 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:25.139114 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49474) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:25.528310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [None req-134efcd5-2d44-40f1-9ed8-1ee18956ee9a admin admin] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:52:25.529360 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-134efcd5-2d44-40f1-9ed8-1ee18956ee9a admin admin] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3892894 May 24 09:52:27.907210 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49484) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:28.294646 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [None req-8b64abd5-f796-4843-b390-cd179349d205 admin admin] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:52:28.297103 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8b64abd5-f796-4843-b390-cd179349d205 admin admin] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3875141 May 24 09:52:29.661623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4ceacdc6-aabd-43b6-81c9-22f276348e93 None None] Network ab4a7426-e455-41e3-a994-4e61d59242c8 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:52:29.794589 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ca0f8129-19e3-48b1-8f0d-d3cae7d93755 None None] Provisioning for port 4fa48412-4c97-42b7-b7d6-55e36371a598 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:52:29.952197 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4fa48412-4c97-42b7-b7d6-55e36371a598 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:30.111842 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 4fa48412-4c97-42b7-b7d6-55e36371a598 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:52:30.127397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 4fa48412-4c97-42b7-b7d6-55e36371a598 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:30.127397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:30.176400 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-83b65947-69cc-4771-8b61-4fe7be8088cf None None] Provisioning complete for port 4fa48412-4c97-42b7-b7d6-55e36371a598 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:30.176777 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-83b65947-69cc-4771-8b61-4fe7be8088cf None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:30.348481 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:30.369122 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-83b65947-69cc-4771-8b61-4fe7be8088cf None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:30.405917 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 4fa48412-4c97-42b7-b7d6-55e36371a598, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4fa48412-4c97-42b7-b7d6-55e36371a598,segment=NetworkSegment(2051bcb0-6c86-4013-9786-b1de425619cb),segment_id=2051bcb0-6c86-4013-9786-b1de425619cb)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 09:52:30.467038 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:30.501680 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:30.578783 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.db.api [None req-83b65947-69cc-4771-8b61-4fe7be8088cf None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=109114) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 24 09:52:30.579154 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_db.api [None req-83b65947-69cc-4771-8b61-4fe7be8088cf None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 24 09:52:30.586427 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=4fa48412-4c97-42b7-b7d6-55e36371a598,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:30.589842 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:30.973568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49500) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:31.340360 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Request body: {'router': {'admin_state_up': True, 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'name': 'router1'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:31.344737 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:31.348434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Usage tracker for resource:router and tenant:33a491aa066c4be7bc85d06159d12d9a is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:31.351682 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:31.357315 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:31.512031 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:31.514183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:31.536122 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:31.718401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:31.814335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:31.815507 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2c17795c-a2cc-4b5b-93fa-87168f26d1a1 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 709 time: 0.8409774 May 24 09:52:31.929454 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4fa48412-4c97-42b7-b7d6-55e36371a598 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:31.998340 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 4fa48412-4c97-42b7-b7d6-55e36371a598 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:31.998718 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:33.300513 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-20bd1ebd-2cf1-4a14-ac7e-69fff93a4ad1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:34.161531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49510) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:34.530022 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [None req-1dcf888b-3ccb-458d-ab36-a929e0f106ca admin admin] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:52:34.531001 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1dcf888b-3ccb-458d-ab36-a929e0f106ca admin admin] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3685620 May 24 09:52:35.746084 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:35.784649 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-be3ea98e-3348-4d71-b73d-5f29a09fa726 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:36.873002 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49520) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:37.123306 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.238087 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Request body: {'network': {'provider:physical_network': 'physnet1', 'is_default': True, 'admin_state_up': True, 'router:external': True, 'provider:network_type': 'flat', 'name': 'public'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:37.244929 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:37.247377 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Usage tracker for resource:network and tenant:ccff31096da8487cb3989249327c9e0c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:37.250319 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:37.256121 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:37.346131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Made reservation on behalf of ccff31096da8487cb3989249327c9e0c for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:52:37.348418 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.367118 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.382093 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:37.385945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Usage tracker for resource:security_group_rule and tenant:ccff31096da8487cb3989249327c9e0c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:37.389750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:37.395588 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:37.488091 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.592290 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.595811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:37.611164 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_flat [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Reserving flat network on physical network physnet1 {{(pid=109113) reserve_provider_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_flat.py:100}} May 24 09:52:37.631694 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6cc5a135-cf97-4966-8e97-5b01de4c83e1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:37.636968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8762070920174', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.638014 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Unsubscribe: segment precommit_create {{(pid=109113) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 24 09:52:37.638458 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Added segment 7bdf7a39-c927-428c-ae01-d27b28cde973 of type flat for network 2211f181-5fdc-4580-9f8a-8610ff86d56a May 24 09:52:37.645015 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:37.678420 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.814378 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.815718 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:37.816294 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:37.821143 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:37.935733 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Pushing event updated for resources: {'Network': ['ID=2211f181-5fdc-4580-9f8a-8610ff86d56a,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:37.938832 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:37.945919 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 09:52:37.975532 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 1.0954716 May 24 09:52:38.648929 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:40.319598 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49540) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:40.537635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f78cf9d8-1a5e-45bb-8d3a-5e79c4f8e632 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:40.539031 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f78cf9d8-1a5e-45bb-8d3a-5e79c4f8e632 admin admin] 10.222.0.67 "GET /v2.0/subnets/9e3a394e-579d-4b2f-a775-1e06acc3e42c HTTP/1.1" status: 200 len: 849 time: 0.2184556 May 24 09:52:40.694993 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-85c87f99-84d6-4194-ac42-74d07f3f05a9 admin admin] 10.222.0.67 "GET /v2.0/routers/83f26990-09f1-49cb-879c-a008d13fbec0 HTTP/1.1" status: 200 len: 704 time: 0.1498575 May 24 09:52:41.163850 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:41.395278 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Allocated IP 172.20.0.1 (ab4a7426-e455-41e3-a994-4e61d59242c8/9e3a394e-579d-4b2f-a775-1e06acc3e42c/1b08d912-0ca2-45cc-a864-c79e2fb93252) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 09:52:41.552905 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:41.570971 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Transition to ACTIVE for port object 1b08d912-0ca2-45cc-a864-c79e2fb93252 will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:52:41.716428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:41.721691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:41.781209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Network ab4a7426-e455-41e3-a994-4e61d59242c8 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 09:52:41.784111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:41.787337 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 09:52:41.787817 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:41.926978 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Pushing event updated for resources: {'Port': ['ID=1b08d912-0ca2-45cc-a864-c79e2fb93252,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:41.930323 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.209s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:42.150883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4e87a8b9-ee41-4dbd-a906-f0af45ce5f75 None None] Provisioning for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:52:42.156419 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4e87a8b9-ee41-4dbd-a906-f0af45ce5f75 None None] Provisioning complete for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:42.156799 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4e87a8b9-ee41-4dbd-a906-f0af45ce5f75 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.195085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.207802 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-4e87a8b9-ee41-4dbd-a906-f0af45ce5f75 None None] Port 1b08d912-0ca2-45cc-a864-c79e2fb93252 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 09:52:42.423111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.506032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.507147 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.511489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:42.648085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Pushing event updated for resources: {'Port': ['ID=1b08d912-0ca2-45cc-a864-c79e2fb93252,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:42.652974 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:42.702251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Network ab4a7426-e455-41e3-a994-4e61d59242c8 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 09:52:42.703313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:42.931301 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-10c5a2cd-bcb7-4c0e-a7a8-f9067352c821 None None] Provisioning complete for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:42.931676 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-10c5a2cd-bcb7-4c0e-a7a8-f9067352c821 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:42.957513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Router 83f26990-09f1-49cb-879c-a008d13fbec0 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 09:52:42.981795 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:52:42.986828 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-10c5a2cd-bcb7-4c0e-a7a8-f9067352c821 None None] Port 1b08d912-0ca2-45cc-a864-c79e2fb93252 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 09:52:43.108537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 09:52:43.117381 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Sync routers for ids ['83f26990-09f1-49cb-879c-a008d13fbec0'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 09:52:43.118439 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] 10.222.0.67 "PUT /v2.0/routers/83f26990-09f1-49cb-879c-a008d13fbec0/add_router_interface HTTP/1.1" status: 200 len: 508 time: 2.4187419 May 24 09:52:43.128035 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Routers {'83f26990-09f1-49cb-879c-a008d13fbec0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 09:52:43.128903 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T09:52:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['83f26990-09f1-49cb-879c-a008d13fbec0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:43.305552 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Set router 83f26990-09f1-49cb-879c-a008d13fbec0 gateway port host: None {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 09:52:43.677357 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:42Z', 'revision_number': 2, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port_host': None, '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:42Z', 'revision_number': 2, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:43.682215 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Checking router: 83f26990-09f1-49cb-879c-a008d13fbec0 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 09:52:43.682578 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Updating router 83f26990-09f1-49cb-879c-a008d13fbec0 port 1b08d912-0ca2-45cc-a864-c79e2fb93252 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 09:52:43.734615 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.126667 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.141740 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Transition to ACTIVE for port object 1b08d912-0ca2-45cc-a864-c79e2fb93252 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:52:44.249026 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.254802 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:44.377562 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7191f517-f908-419f-8902-d17577593737 None None] Pushing event updated for resources: {'Port': ['ID=1b08d912-0ca2-45cc-a864-c79e2fb93252,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:44.414580 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.414922 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.415249 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.415568 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.415881 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.416194 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.416502 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.416811 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.417154 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.417468 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.417771 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.418076 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7191f517-f908-419f-8902-d17577593737 None None] Tracked resource versions report: May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: pushed versions: May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: { 'AddressGroup': {'1.2'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'ConntrackHelper': {'1.0'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Log': {'1.0'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Network': {'1.1'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Port': {'1.7'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'PortForwarding': {'1.2'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'QosPolicy': {'1.8'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroup': {'1.4'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroupRule': {'1.2'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SubPort': {'1.0'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Subnet': {'1.1'}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Trunk': {'1.1'}} May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: consumer versions: May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'ConntrackHelper': '1.0', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Log': '1.0', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Network': '1.1', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Port': '1.7', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'PortForwarding': '1.2', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'QosPolicy': '1.8', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroup': '1.4', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroupRule': '1.2', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SubPort': '1.0', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Subnet': '1.1', May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Trunk': '1.1'}} May 24 09:52:44.419209 dv-cinder-791239-2-iscsi neutron-server[109115]: {{(pid=109115) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 09:52:44.429879 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7191f517-f908-419f-8902-d17577593737 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:44.488282 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Attempting to bind port 1b08d912-0ca2-45cc-a864-c79e2fb93252 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 09:52:44.488683 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Attempting to bind port 1b08d912-0ca2-45cc-a864-c79e2fb93252 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 09:52:44.489116 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Attempting to bind port 1b08d912-0ca2-45cc-a864-c79e2fb93252 on network ab4a7426-e455-41e3-a994-4e61d59242c8 {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 09:52:44.494373 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 9, 52, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 09:52:44.494787 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Checking segment: {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:52:44.495252 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.495639 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Bound using segment: {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 09:52:44.496149 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Bound port: 1b08d912-0ca2-45cc-a864-c79e2fb93252, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '2051bcb0-6c86-4013-9786-b1de425619cb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 70, 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 09:52:44.496515 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.588930 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] For port 1b08d912-0ca2-45cc-a864-c79e2fb93252, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 09:52:44.593008 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] For port 1b08d912-0ca2-45cc-a864-c79e2fb93252, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=1b08d912-0ca2-45cc-a864-c79e2fb93252,segment=NetworkSegment(2051bcb0-6c86-4013-9786-b1de425619cb),segment_id=2051bcb0-6c86-4013-9786-b1de425619cb)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 09:52:44.757850 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:44.761822 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7191f517-f908-419f-8902-d17577593737 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:44.871444 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7191f517-f908-419f-8902-d17577593737 None None] Pushing event updated for resources: {'Port': ['ID=1b08d912-0ca2-45cc-a864-c79e2fb93252,revision_number=4']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:44.876403 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7191f517-f908-419f-8902-d17577593737 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:44.877161 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Routers {'83f26990-09f1-49cb-879c-a008d13fbec0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 09:52:44.877761 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T09:52:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['83f26990-09f1-49cb-879c-a008d13fbec0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:45.030895 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] Set router 83f26990-09f1-49cb-879c-a008d13fbec0 gateway port host: None {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 09:52:45.291051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49562) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:45.365029 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:42Z', 'revision_number': 2, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port_host': None, '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:44Z', 'revision_number': 4, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:45.369175 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3334971-46b1-4417-81a4-ac3c5f240b82 None None] The sync data for ids ['83f26990-09f1-49cb-879c-a008d13fbec0'] in dv-cinder-791239-2-iscsi is: [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:42Z', 'revision_number': 2, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port_host': None, '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:44Z', 'revision_number': 4, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 09:52:45.446241 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c3c15f37-ddd4-4172-931b-558f42b5ef4e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:45.447605 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c3c15f37-ddd4-4172-931b-558f42b5ef4e admin admin] 10.222.0.67 "GET /v2.0/networks/2211f181-5fdc-4580-9f8a-8610ff86d56a HTTP/1.1" status: 200 len: 842 time: 0.1556783 May 24 09:52:45.456730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Request body: {'subnet': {'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'enable_dhcp': False, 'name': 'public-subnet', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:45.461944 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:52:45.466431 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Usage tracker for resource:subnet and tenant:ccff31096da8487cb3989249327c9e0c is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:52:45.469419 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:52:45.475123 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:52:45.620559 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Made reservation on behalf of ccff31096da8487cb3989249327c9e0c for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:52:45.622933 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:46.148826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.objects.ports [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Time-cost: Fetching 0 ports in 0 security groups {{(pid=109112) get_objects /opt/stack/neutron/neutron/objects/ports.py:443}} May 24 09:52:46.149220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:46.153596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:46.264277 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Pushing event updated for resources: {'Subnet': ['ID=c29e3891-6552-402c-9c30-4a0d6fd7e6a6,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:46.270732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:46.273508 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:52:46.273930 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Network hostable DHCP agents. Network: 2211f181-5fdc-4580-9f8a-8610ff86d56a, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 09:52:46.448768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 09:52:46.608648 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 2211f181-5fdc-4580-9f8a-8610ff86d56a) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:46.612904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:46.613547 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:46.617182 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d17bc3a6-98c0-4cd6-b0ff-1d52cf264921 None None] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:52:46.751066 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 821 time: 1.2977228 May 24 09:52:46.815572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-07a3727b-a4ec-4a30-8ed0-d41854308f68 None None] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:52:48.002665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 1b08d912-0ca2-45cc-a864-c79e2fb93252 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:48.155830 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:52:48.160990 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:48.161432 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:48.294687 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:48.327808 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 1b08d912-0ca2-45cc-a864-c79e2fb93252, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=1b08d912-0ca2-45cc-a864-c79e2fb93252,segment=NetworkSegment(2051bcb0-6c86-4013-9786-b1de425619cb),segment_id=2051bcb0-6c86-4013-9786-b1de425619cb)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 09:52:48.458955 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:48.461308 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:48.529634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=1b08d912-0ca2-45cc-a864-c79e2fb93252,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:48.536757 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:49.018646 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 49576) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:49.217423 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-eef3c31b-c7a8-401f-9fe9-e1aaaa33be89 admin admin] 10.222.0.67 "GET /v2.0/routers/83f26990-09f1-49cb-879c-a008d13fbec0 HTTP/1.1" status: 200 len: 710 time: 0.1975324 May 24 09:52:49.346270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-57a5d121-4d4e-4002-8215-7d20f134418a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:52:49.347653 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-57a5d121-4d4e-4002-8215-7d20f134418a admin admin] 10.222.0.67 "GET /v2.0/networks/2211f181-5fdc-4580-9f8a-8610ff86d56a HTTP/1.1" status: 200 len: 886 time: 0.1252706 May 24 09:52:49.356251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Request body: {'router': {'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:52:49.790368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:49.941184 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 1b08d912-0ca2-45cc-a864-c79e2fb93252 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:50.014295 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:50.014664 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:50.194140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:50.303037 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:50.432841 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Allocated IP 172.27.0.161 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/d6872b22-5e95-49de-a46e-f9b344e26e68) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 09:52:50.600743 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:50.723373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:50.727191 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:50.786003 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 09:52:50.788782 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:52:50.794875 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 09:52:50.850577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Pushing event updated for resources: {'Port': ['ID=d6872b22-5e95-49de-a46e-f9b344e26e68,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:50.860567 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:51.248359 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:51.591115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:51.594390 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_distributed_port_bindings_after_change-425458', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_snat_interfaces_after_change-425395', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._reconfigure_ha_resources-423207'] for router, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:51.769274 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Router 83f26990-09f1-49cb-879c-a008d13fbec0 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 09:52:51.993783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 09:52:52.002588 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:52.003639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Sync routers for ids ['83f26990-09f1-49cb-879c-a008d13fbec0'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 09:52:52.004093 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] 10.222.0.67 "PUT /v2.0/routers/83f26990-09f1-49cb-879c-a008d13fbec0 HTTP/1.1" status: 200 len: 890 time: 2.6507680 May 24 09:52:52.014493 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Routers {'83f26990-09f1-49cb-879c-a008d13fbec0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 09:52:52.015189 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T09:52:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['83f26990-09f1-49cb-879c-a008d13fbec0'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:52.207829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Set router 83f26990-09f1-49cb-879c-a008d13fbec0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 09:52:52.571903 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161'}], 'enable_snat': True}, 'gw_port_id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:50Z', 'revision_number': 4, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port': {'id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:44:0c:d2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:50Z', 'updated_at': '2021-05-24T09:52:50Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:48Z', 'revision_number': 5, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:52.577585 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Checking router: 83f26990-09f1-49cb-879c-a008d13fbec0 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 09:52:52.578112 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Updating router 83f26990-09f1-49cb-879c-a008d13fbec0 port d6872b22-5e95-49de-a46e-f9b344e26e68 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 09:52:52.631168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:52.964422 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:52.977073 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Transition to ACTIVE for port object d6872b22-5e95-49de-a46e-f9b344e26e68 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:52:53.077010 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:53.082516 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-a1a815df-cd9a-4332-bcd4-120841fb6cd9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:53.160173 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Attempting to bind port d6872b22-5e95-49de-a46e-f9b344e26e68 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 09:52:53.160554 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Attempting to bind port d6872b22-5e95-49de-a46e-f9b344e26e68 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 09:52:53.160933 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Attempting to bind port d6872b22-5e95-49de-a46e-f9b344e26e68 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 09:52:53.166623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Pushing event updated for resources: {'Port': ['ID=d6872b22-5e95-49de-a46e-f9b344e26e68,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:53.170675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:53.171815 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 9, 52, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 09:52:53.172196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:52:53.172603 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:53.172996 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 09:52:53.174280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Bound port: d6872b22-5e95-49de-a46e-f9b344e26e68, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 09:52:53.174280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:53.248078 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] For port d6872b22-5e95-49de-a46e-f9b344e26e68, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 09:52:53.251145 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] For port d6872b22-5e95-49de-a46e-f9b344e26e68, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=d6872b22-5e95-49de-a46e-f9b344e26e68,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 09:52:53.368994 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:53.373854 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:53.453391 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Routers {'83f26990-09f1-49cb-879c-a008d13fbec0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 09:52:53.454023 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T09:52:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['83f26990-09f1-49cb-879c-a008d13fbec0'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:53.501999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Pushing event updated for resources: {'Port': ['ID=d6872b22-5e95-49de-a46e-f9b344e26e68,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:53.509668 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:53.638111 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] Set router 83f26990-09f1-49cb-879c-a008d13fbec0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 09:52:53.967634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161'}], 'enable_snat': True}, 'gw_port_id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:50Z', 'revision_number': 4, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port': {'id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:44:0c:d2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:50Z', 'updated_at': '2021-05-24T09:52:53Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:48Z', 'revision_number': 5, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 09:52:53.971630 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fbc1d6f1-8c5a-4179-b0d1-1704d3e4871c None None] The sync data for ids ['83f26990-09f1-49cb-879c-a008d13fbec0'] in dv-cinder-791239-2-iscsi is: [{'id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'name': 'router1', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161'}], 'enable_snat': True}, 'gw_port_id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T09:52:31Z', 'updated_at': '2021-05-24T09:52:50Z', 'revision_number': 4, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'gw_port': {'id': 'd6872b22-5e95-49de-a46e-f9b344e26e68', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:44:0c:d2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.161', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:50Z', 'updated_at': '2021-05-24T09:52:53Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '1b08d912-0ca2-45cc-a864-c79e2fb93252', 'name': '', 'network_id': 'ab4a7426-e455-41e3-a994-4e61d59242c8', 'tenant_id': '33a491aa066c4be7bc85d06159d12d9a', 'mac_address': 'fa:16:3e:92:56:3c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '83f26990-09f1-49cb-879c-a008d13fbec0', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T09:52:41Z', 'updated_at': '2021-05-24T09:52:48Z', 'revision_number': 5, 'project_id': '33a491aa066c4be7bc85d06159d12d9a', 'subnets': [{'id': '9e3a394e-579d-4b2f-a775-1e06acc3e42c', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '5f5518c3-256f-4498-8245-e4e8406cf4e1'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 09:52:54.387244 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 49586) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:52:54.544993 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1cf2d30a-f6ab-4f88-9e95-83264012b0b3 admin admin] 10.222.0.67 "GET /v2.0/ports?device_owner=network%3Arouter_gateway&fields=id&fields=name&fields=mac_address&fields=fixed_ips&fields=status HTTP/1.1" status: 200 len: 411 time: 0.1566401 May 24 09:52:55.967073 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d6872b22-5e95-49de-a46e-f9b344e26e68 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:56.117001 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port d6872b22-5e95-49de-a46e-f9b344e26e68 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:52:56.122306 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port d6872b22-5e95-49de-a46e-f9b344e26e68 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:56.124007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:56.283362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:56.300411 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port d6872b22-5e95-49de-a46e-f9b344e26e68, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=d6872b22-5e95-49de-a46e-f9b344e26e68,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 09:52:56.380153 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:52:56.382395 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:52:56.454191 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=d6872b22-5e95-49de-a46e-f9b344e26e68,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:52:56.457778 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:52:57.975465 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d6872b22-5e95-49de-a46e-f9b344e26e68 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:52:58.056054 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port d6872b22-5e95-49de-a46e-f9b344e26e68 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:52:58.056378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:03.271642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7bc3b40c-c6c6-4158-90ba-0237f82164b1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:05.799454 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:05.843201 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c1f0bc96-b2b7-4804-9af9-5b1d4593dc93 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:07.123034 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:08.665780 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:19.981648 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:53:33.287793 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-0a00d4f7-f977-4e6a-813d-5a7917722731 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:35.837675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:35.888109 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-52a3a926-65e9-4c3b-873c-098edc3cd2ed None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:37.131760 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:39.116391 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:53:56.980839 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:54:03.259371 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-114d69fe-ec33-4153-8905-4498ded624da None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:05.778740 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:05.795548 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e4f29c3c-c9b1-4ae6-8082-074ff236a646 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:07.138635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:08.692831 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:31.777220 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50156) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:31.844315 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.pecan_wsgi.hooks.translation [None req-560764f8-780d-439b-9c43-b9c665125442 admin admin] GET failed (client error): The resource could not be found. May 24 09:54:31.844826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-560764f8-780d-439b-9c43-b9c665125442 admin admin] Synchronizing usage tracker for tenant:ccff31096da8487cb3989249327c9e0c on resource:network {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 09:54:31.932263 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-560764f8-780d-439b-9c43-b9c665125442 admin admin] Unset dirty status for tenant:ccff31096da8487cb3989249327c9e0c on resource:network {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 09:54:31.933963 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-560764f8-780d-439b-9c43-b9c665125442 admin admin] 10.222.0.67 "GET /v2.0/networks/private HTTP/1.1" status: 404 len: 315 time: 0.1558092 May 24 09:54:32.065886 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-207bb23b-4331-46e0-bc48-919380657c24 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:32.067263 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-207bb23b-4331-46e0-bc48-919380657c24 admin admin] 10.222.0.67 "GET /v2.0/networks?name=private HTTP/1.1" status: 200 len: 871 time: 0.1294200 May 24 09:54:33.291235 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-64c9f220-d895-426d-a8c0-4c5043bb14ef None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:33.979703 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:54:34.165147 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50166) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:34.274603 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-db5f7aad-ba6a-4ec8-a5f9-880b320288c6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:34.275941 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-db5f7aad-ba6a-4ec8-a5f9-880b320288c6 admin admin] 10.222.0.67 "GET /v2.0/subnets/9e3a394e-579d-4b2f-a775-1e06acc3e42c HTTP/1.1" status: 200 len: 849 time: 0.1098654 May 24 09:54:34.285583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Request body: {'subnet': {'dns_nameservers': ['8.8.8.8', '8.8.4.4']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:54:34.401295 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Notify callbacks [] for subnet, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:34.737934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_subnet_gateway_ip_update-8762070893469'] for subnet, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:34.741888 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-69150663-293c-4870-9bc4-19aa37280ca6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:54:34.782486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:34.783500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:34.784315 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] 10.222.0.67 "PUT /v2.0/subnets/9e3a394e-579d-4b2f-a775-1e06acc3e42c HTTP/1.1" status: 200 len: 868 time: 0.5024025 May 24 09:54:34.800962 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c9276a6f-87a8-43d8-893e-ad6ffcd51c7b None None] Network ab4a7426-e455-41e3-a994-4e61d59242c8 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:54:34.810507 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Pushing event updated for resources: {'Subnet': ['ID=9e3a394e-579d-4b2f-a775-1e06acc3e42c,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:54:34.818743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.819094 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.819423 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.819751 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.820072 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.820902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.820902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.821107 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.821379 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.821699 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.822012 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.822375 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:54:34.826692 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:54:35.208615 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c9c550e-3b18-47c6-9bcb-09f6819ebe1a None None] Provisioning complete for port 4fa48412-4c97-42b7-b7d6-55e36371a598 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:54:35.208986 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c9c550e-3b18-47c6-9bcb-09f6819ebe1a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:35.334668 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c9c550e-3b18-47c6-9bcb-09f6819ebe1a None None] Provisioning complete for port 1b08d912-0ca2-45cc-a864-c79e2fb93252 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:54:35.334927 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c9c550e-3b18-47c6-9bcb-09f6819ebe1a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:35.784149 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:35.852448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d6b2cf3e-b401-479f-8644-092a9c0a4949 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:36.912048 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50176) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:37.087670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1f6b4f1a-7395-4623-8a95-de5d8988f55f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:37.088131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1f6b4f1a-7395-4623-8a95-de5d8988f55f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:37.089678 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1f6b4f1a-7395-4623-8a95-de5d8988f55f admin admin] 10.222.0.67 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1548 time: 0.1766944 May 24 09:54:37.155482 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:38.669887 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:54:39.206143 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50186) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:39.378314 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-47ba6909-4444-4d82-ad7d-c0926d203497 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:39.378783 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-47ba6909-4444-4d82-ad7d-c0926d203497 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:39.382138 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-47ba6909-4444-4d82-ad7d-c0926d203497 admin admin] 10.222.0.67 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1548 time: 0.1732211 May 24 09:54:41.504034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50196) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:41.672955 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-be6dad17-f6e3-44d2-877f-87e162810c16 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:41.674331 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-be6dad17-f6e3-44d2-877f-87e162810c16 admin admin] 10.222.0.67 "GET /v2.0/networks/2211f181-5fdc-4580-9f8a-8610ff86d56a HTTP/1.1" status: 200 len: 886 time: 0.1693609 May 24 09:54:43.790074 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50206) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:43.969326 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dc6c91bd-c30f-4685-96df-3dd05f6e20f1 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:43.970686 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-dc6c91bd-c30f-4685-96df-3dd05f6e20f1 admin admin] 10.222.0.67 "GET /v2.0/networks/ab4a7426-e455-41e3-a994-4e61d59242c8 HTTP/1.1" status: 200 len: 868 time: 0.1797225 May 24 09:54:46.078138 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50216) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:46.188135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7cd89d5e-1f20-4b90-ba75-e15ba915cb46 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:46.188597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7cd89d5e-1f20-4b90-ba75-e15ba915cb46 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:46.190150 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7cd89d5e-1f20-4b90-ba75-e15ba915cb46 admin admin] 10.222.0.67 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1110606 May 24 09:54:48.305603 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50226) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:48.411905 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0251cfba-5c81-4013-9cbf-37d1a7ca7ac2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:48.412329 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0251cfba-5c81-4013-9cbf-37d1a7ca7ac2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:48.413878 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0251cfba-5c81-4013-9cbf-37d1a7ca7ac2 admin admin] 10.222.0.67 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1073523 May 24 09:54:50.554813 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50236) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:50.663520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b47d5c76-8eec-41ea-b4b9-f19d3e88e1f6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:50.664080 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b47d5c76-8eec-41ea-b4b9-f19d3e88e1f6 admin admin] Synchronizing usage tracker for tenant:ccff31096da8487cb3989249327c9e0c on resource:subnet {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 09:54:50.783349 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b47d5c76-8eec-41ea-b4b9-f19d3e88e1f6 admin admin] Unset dirty status for tenant:ccff31096da8487cb3989249327c9e0c on resource:subnet {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 09:54:50.784548 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b47d5c76-8eec-41ea-b4b9-f19d3e88e1f6 admin admin] 10.222.0.67 "GET /v2.0/subnets/9e3a394e-579d-4b2f-a775-1e06acc3e42c HTTP/1.1" status: 200 len: 868 time: 0.2287955 May 24 09:54:52.884842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50246) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:54:52.984052 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-75e36956-0145-4521-a565-04614069cc72 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:54:52.985766 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-75e36956-0145-4521-a565-04614069cc72 admin admin] 10.222.0.67 "GET /v2.0/subnets/c29e3891-6552-402c-9c30-4a0d6fd7e6a6 HTTP/1.1" status: 200 len: 816 time: 0.0995831 May 24 09:55:03.286975 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a4bed1d5-7780-4ef0-aaf4-dc5140de2ae2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:05.743704 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:05.777690 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1cfbfcaa-480c-4309-a009-087f3ae3eab1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:07.172343 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:08.646092 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:10.978166 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:55:16.178594 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50290) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:55:16.232211 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [None req-f367e4d4-805e-4527-88af-fa599eb7848c admin admin] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109112) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:55:16.233284 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f367e4d4-805e-4527-88af-fa599eb7848c admin admin] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0536962 May 24 09:55:18.454091 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50300) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:55:18.523806 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.pecan_wsgi.hooks.translation [None req-649f1d40-6a35-4275-a9e9-7f33c5ed429e admin admin] GET failed (client error): The resource could not be found. May 24 09:55:18.525646 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-649f1d40-6a35-4275-a9e9-7f33c5ed429e admin admin] 10.222.0.67 "GET /v2.0/networks/public HTTP/1.1" status: 404 len: 314 time: 0.0701528 May 24 09:55:18.657073 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1ed4a517-f29a-4cc0-a84e-e1be6c443f21 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:55:18.658454 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1ed4a517-f29a-4cc0-a84e-e1be6c443f21 admin admin] 10.222.0.67 "GET /v2.0/networks?name=public HTTP/1.1" status: 200 len: 889 time: 0.1281958 May 24 09:55:20.821591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50310) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:55:20.873653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Request body: {'network': {'name': 'shared', 'shared': True, 'admin_state_up': True}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:55:20.878424 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:55:20.888623 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Attempting to reserve 1 items for resource network. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:55:20.954719 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Made reservation on behalf of ccff31096da8487cb3989249327c9e0c for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:55:20.957070 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:21.075455 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING neutron.services.revisions.revision_plugin [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Could not find related network for resource to bump revision. May 24 09:55:21.087310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 54} {{(pid=109113) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 09:55:21.096697 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:21.097035 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Added segment 34dfaf71-8e80-44a9-8573-f269e60764b0 of type vlan for network 2354deba-3cdc-41e6-b0d7-618ece22224b May 24 09:55:21.122198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:21.211843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:21.213215 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:55:21.213796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:21.217266 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:55:21.318127 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Pushing event updated for resources: {'Network': ['ID=2354deba-3cdc-41e6-b0d7-618ece22224b,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:55:21.321103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:55:21.331986 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 2354deba-3cdc-41e6-b0d7-618ece22224b details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 09:55:21.405563 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 846 time: 0.5828526 May 24 09:55:23.594630 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50320) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:55:23.665403 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.pecan_wsgi.hooks.translation [None req-cc25bae1-09ab-4746-bede-34982893760b admin admin] GET failed (client error): The resource could not be found. May 24 09:55:23.665929 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cc25bae1-09ab-4746-bede-34982893760b admin admin] Synchronizing usage tracker for tenant:ccff31096da8487cb3989249327c9e0c on resource:network {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 09:55:23.783128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cc25bae1-09ab-4746-bede-34982893760b admin admin] Unset dirty status for tenant:ccff31096da8487cb3989249327c9e0c on resource:network {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 09:55:23.784209 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-cc25bae1-09ab-4746-bede-34982893760b admin admin] 10.222.0.67 "GET /v2.0/networks/shared HTTP/1.1" status: 404 len: 314 time: 0.1886196 May 24 09:55:23.887219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-4e1f0c8f-c6f6-4dfb-b124-bcfc7c5385f6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:55:23.888704 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-4e1f0c8f-c6f6-4dfb-b124-bcfc7c5385f6 admin admin] 10.222.0.67 "GET /v2.0/networks?name=shared HTTP/1.1" status: 200 len: 825 time: 0.1005332 May 24 09:55:23.898587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Request body: {'subnet': {'cidr': '192.168.233.0/24', 'ip_version': 4, 'name': 'shared-subnet', 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b', 'description': 'shared-subnet'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 09:55:23.903255 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:55:23.912746 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:55:24.011571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Made reservation on behalf of ccff31096da8487cb3989249327c9e0c for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 09:55:24.013979 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:24.478211 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:24.482349 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:55:24.550883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4454f008-b32d-425f-94d3-001f5811f698,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:55:24.555667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:55:24.564593 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Checking segment: {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:55:24.564942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Network hostable DHCP agents. Network: 2354deba-3cdc-41e6-b0d7-618ece22224b, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 09:55:24.696001 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Network 2354deba-3cdc-41e6-b0d7-618ece22224b is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 09:55:24.776291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 2354deba-3cdc-41e6-b0d7-618ece22224b) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:55:24.781237 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 09:55:24.781821 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:24.788430 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3bc3824f-06e6-43d9-868b-94cce4e1ec11 None None] Network 2354deba-3cdc-41e6-b0d7-618ece22224b requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:55:24.898168 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.0029883 May 24 09:55:25.030116 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-2354deba-3cdc-41e6-b0d7-618ece22224b', 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b', 'tenant_id': 'ccff31096da8487cb3989249327c9e0c', 'fixed_ips': [{'subnet_id': '4454f008-b32d-425f-94d3-001f5811f698'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 09:55:25.030880 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.189845 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Allocated IP 192.168.233.2 (2354deba-3cdc-41e6-b0d7-618ece22224b/4454f008-b32d-425f-94d3-001f5811f698/487509a0-d742-404d-aa6e-9aa5e036b16c) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 09:55:25.401856 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.412337 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Transition to ACTIVE for port object 487509a0-d742-404d-aa6e-9aa5e036b16c will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:55:25.432081 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Transition to ACTIVE for port object 487509a0-d742-404d-aa6e-9aa5e036b16c will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 09:55:25.503483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.508350 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:55:25.576856 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Network 2354deba-3cdc-41e6-b0d7-618ece22224b is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 09:55:25.578074 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 09:55:25.581872 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 09:55:25.590048 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Pushing event updated for resources: {'Port': ['ID=487509a0-d742-404d-aa6e-9aa5e036b16c,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:55:25.628688 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.629060 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.629415 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.629743 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.630068 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.630402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.630721 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.631045 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.631378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.631780 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.632089 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.632401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:55:25.640933 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:55:25.693352 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Attempting to bind port 487509a0-d742-404d-aa6e-9aa5e036b16c on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 09:55:25.693636 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Attempting to bind port 487509a0-d742-404d-aa6e-9aa5e036b16c by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 09:55:25.693991 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Attempting to bind port 487509a0-d742-404d-aa6e-9aa5e036b16c on network 2354deba-3cdc-41e6-b0d7-618ece22224b {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 09:55:25.699387 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 9, 55, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 09:55:25.699750 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Checking segment: {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 09:55:25.700141 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.700523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Bound using segment: {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 09:55:25.700977 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Bound port: 487509a0-d742-404d-aa6e-9aa5e036b16c, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 09:55:25.701362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.790259 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] For port 487509a0-d742-404d-aa6e-9aa5e036b16c, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 09:55:25.793992 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] For port 487509a0-d742-404d-aa6e-9aa5e036b16c, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=487509a0-d742-404d-aa6e-9aa5e036b16c,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 09:55:25.880519 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:25.885234 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:55:25.964999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Pushing event updated for resources: {'Port': ['ID=487509a0-d742-404d-aa6e-9aa5e036b16c,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:55:25.968853 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:55:26.742384 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f885d02d-47db-43f0-8228-ba30c2e14de2 None None] Network 2354deba-3cdc-41e6-b0d7-618ece22224b requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 09:55:26.905892 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-025cbe6b-78e1-4400-94a5-9b228050ab61 None None] Provisioning for port 487509a0-d742-404d-aa6e-9aa5e036b16c completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:55:28.079608 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 487509a0-d742-404d-aa6e-9aa5e036b16c up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:55:28.227308 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 487509a0-d742-404d-aa6e-9aa5e036b16c completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 09:55:28.241304 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 487509a0-d742-404d-aa6e-9aa5e036b16c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:55:28.241775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:28.360032 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:28.392307 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 487509a0-d742-404d-aa6e-9aa5e036b16c, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=487509a0-d742-404d-aa6e-9aa5e036b16c,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 09:55:28.463169 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:28.465305 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:55:28.531426 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=487509a0-d742-404d-aa6e-9aa5e036b16c,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:55:28.534909 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:55:30.054783 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 487509a0-d742-404d-aa6e-9aa5e036b16c up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 09:55:30.127789 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 487509a0-d742-404d-aa6e-9aa5e036b16c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 09:55:30.128151 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:33.291676 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-71c245c8-1f4b-439a-9a66-a0cce967f47a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:35.741558 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:35.771379 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a85d788e-57dd-4ecc-8445-894e48e18723 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:37.119918 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:38.657015 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:55:47.977157 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:56:03.320175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4d23bcb6-fe60-424c-bd9c-bf35313a0b20 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:05.737021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:05.759739 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-57c56fb6-5cc4-4fae-9094-0143d3adbb7a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:07.170724 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:08.673666 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:24.976752 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:56:33.261985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e795e07c-38e3-4b48-bc2b-c4f5068fd5ac None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:35.758987 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:35.781832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-bdc55851-147b-4063-8e13-c77a8f3df9ce None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:37.147194 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:56:38.873106 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:01.976405 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:57:03.262071 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-f5411fd8-41ae-445b-b55e-776e63662db7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:05.768565 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:05.803626 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-74e0b7c8-57a3-47dd-8a5d-de7c0f84fca8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:07.158775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:08.702131 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:13.950863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50370) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:57:13.956862 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [None req-2cbc1ccb-b81a-465b-9ebe-676004fb001f tempest-verify_tempest_config-449577301 tempest-verify_tempest_config-449577301-project] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 09:57:13.957904 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-2cbc1ccb-b81a-465b-9ebe-676004fb001f tempest-verify_tempest_config-449577301 tempest-verify_tempest_config-449577301-project] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13635 time: 0.0060835 May 24 09:57:14.526060 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50388) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:57:14.651854 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:14.671462 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 09:57:14.675582 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Usage tracker for resource:security_group_rule and tenant:d215c80cf8564479942e05d2fd92cef3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 09:57:14.678593 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 09:57:14.685266 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 09:57:14.779443 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:14.930017 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:14.938657 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:57:14.982719 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=d215c80cf8564479942e05d2fd92cef3&name=default HTTP/1.1" status: 200 len: 2975 time: 0.4554608 May 24 09:57:14.983702 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5d2067d5-cfad-4bfa-99b7-182f734b9bbd,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:57:14.985973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50394) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:57:14.991378 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.991726 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.992066 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.992401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.992730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.993111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.993451 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.993793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.994142 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.994465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.994786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.995105 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:14.999219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:57:15.096494 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:15.144669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:15.380808 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:15.385454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 09:57:15.398604 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5d2067d5-cfad-4bfa-99b7-182f734b9bbd,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 09:57:15.407891 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.408261 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.408603 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.408935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.409293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.409638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.409969 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.410312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.410656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.410979 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.411293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.411614 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 09:57:15.415014 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 09:57:15.416972 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:15.423247 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/5d2067d5-cfad-4bfa-99b7-182f734b9bbd HTTP/1.1" status: 204 len: 168 time: 0.4342136 May 24 09:57:33.306544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3fddc1c8-9c72-4e79-9cce-270abe7ee3f1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:35.768254 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:35.780865 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6743ce00-e8c3-4e51-b1d0-67dcdc165acf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:37.175627 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:38.678025 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:57:38.976111 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:58:03.289144 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-bc80c1d2-36ea-4533-9c89-0bd010624a2f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:05.780127 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:05.791836 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2f7f183b-9239-448b-917f-f427e7355dbf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:07.140528 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:08.717530 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:15.976082 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 5 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:58:33.270996 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8e646c53-70a5-4c55-b9b4-d58a57818136 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:35.790366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:35.818581 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5c7eca1-0a7f-45d5-9617-877e173357fc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:37.168720 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:38.710912 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:50.018221 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183'] for agent, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:58:52.975049 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:58:53.975741 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50446) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:58:54.087742 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f6986a67-e6b8-4a08-8751-6bc7182f5e89 admin admin] 10.222.0.67 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.1109302 May 24 09:58:56.249779 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50456) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 09:58:56.312396 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-95e586ea-97fd-4585-be41-c1de86d5ba50 admin admin] 10.222.0.67 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.0611899 May 24 09:59:03.276597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-576aad5b-6fd1-4c10-a1d2-790dbd5bfc1a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:05.808108 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:05.842456 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-db8a4f65-eda0-4a24-8d51-69a1e6c1d41f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:07.144965 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:08.744954 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:19.996339 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:29.974348 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 09:59:33.275678 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c82aa4c0-c6f8-4685-9c67-6190c5441643 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:35.767467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:35.784470 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fe39e7fa-5505-4874-b426-0d10712df669 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:37.138228 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:38.704597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 09:59:49.981207 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:03.263766 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ffb3b5d3-bee8-44d3-963f-d4861c5c5746 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:05.744138 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:05.767094 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-502bbefd-d758-4a59-9061-b09a10c67b42 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:06.974345 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:00:07.143983 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:08.698297 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:20.006809 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.008672 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50628) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:23.400943 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Request body: {'network': {'name': 'tempest-DeleteServersTestJSON-1130545244-network', 'tenant_id': '35a01011c3994805ba2a956d0daf4aa6'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:23.406611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:23.410951 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Usage tracker for resource:network and tenant:35a01011c3994805ba2a956d0daf4aa6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:23.414469 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:23.420769 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:23.526206 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Made reservation on behalf of 35a01011c3994805ba2a956d0daf4aa6 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:23.528632 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.541287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.556543 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:23.561140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Usage tracker for resource:security_group_rule and tenant:35a01011c3994805ba2a956d0daf4aa6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:23.564032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:23.580745 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:23.666773 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.743746 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.747699 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:23.778876 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 55} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:00:23.784560 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.799768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Tracked resource versions report: May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: pushed versions: May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: { 'AddressGroup': {'1.2'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'ConntrackHelper': {'1.0'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Log': {'1.0'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Network': {'1.1'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Port': {'1.7'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'PortForwarding': {'1.2'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'QosPolicy': {'1.8'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroup': {'1.4'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroupRule': {'1.2'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SubPort': {'1.0'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Subnet': {'1.1'}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Trunk': {'1.1'}} May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: consumer versions: May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: HyperV agent@hv-cinder-79123: {}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'ConntrackHelper': '1.0', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Log': '1.0', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Network': '1.1', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Port': '1.7', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'PortForwarding': '1.2', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'QosPolicy': '1.8', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroup': '1.4', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SecurityGroupRule': '1.2', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'SubPort': '1.0', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Subnet': '1.1', May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: 'Trunk': '1.1'}} May 24 10:00:23.800811 dv-cinder-791239-2-iscsi neutron-server[109112]: {{(pid=109112) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 10:00:23.804206 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:23.817452 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:23.822826 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Added segment deba89ad-e4b9-43a6-b2c3-6904c93466bf of type vlan for network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 May 24 10:00:23.845610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:24.080703 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:24.082096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:24.082725 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:24.087513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:24.195098 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Pushing event updated for resources: {'Network': ['ID=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:24.195098 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:24.200085 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:00:24.285180 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50680) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:24.285732 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 1.2758286 May 24 10:00:24.292532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50684) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:24.294273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Request body: {'subnet': {'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138', 'cidr': '172.20.0.0/28', 'name': 'tempest-DeleteServersTestJSON-1130545244-subnet', 'tenant_id': '35a01011c3994805ba2a956d0daf4aa6', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:24.299513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:24.306273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Usage tracker for resource:subnet and tenant:35a01011c3994805ba2a956d0daf4aa6 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:24.312564 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:24.319019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:24.424151 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Made reservation on behalf of 35a01011c3994805ba2a956d0daf4aa6 for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:24.427300 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:24.525167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50702) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:24.853367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-network', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:24.864632 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:24.873816 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Usage tracker for resource:network and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:24.879440 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:24.892132 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:24.892937 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:24.897584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-05c776fa-493c-40ca-af80-1d3bea3cc14a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:24.917390 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeTestJSON-127484096-network', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:24.926498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:24.934075 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Usage tracker for resource:network and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:24.939255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:24.949348 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:24.970849 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Pushing event updated for resources: {'Subnet': ['ID=b28ebd86-e35c-4b85-b6ba-3336770f64e8,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:24.986227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.986573 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.986908 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.987523 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.987761 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.987936 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.988298 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.988522 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.988904 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.989257 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.989576 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.989890 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Tracked resource versions report: May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: pushed versions: May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: { 'AddressGroup': {'1.2'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'ConntrackHelper': {'1.0'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Log': {'1.0'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Network': {'1.1'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Port': {'1.7'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'PortForwarding': {'1.2'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'QosPolicy': {'1.8'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroup': {'1.4'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroupRule': {'1.2'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SubPort': {'1.0'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Subnet': {'1.1'}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Trunk': {'1.1'}} May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: consumer versions: May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: HyperV agent@hv-cinder-79123: {}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'ConntrackHelper': '1.0', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Log': '1.0', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Network': '1.1', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Port': '1.7', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'PortForwarding': '1.2', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'QosPolicy': '1.8', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroup': '1.4', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SecurityGroupRule': '1.2', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'SubPort': '1.0', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Subnet': '1.1', May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: 'Trunk': '1.1'}} May 24 10:00:24.990879 dv-cinder-791239-2-iscsi neutron-server[109113]: {{(pid=109113) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 10:00:24.999092 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:24.999807 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Checking segment: {'id': 'deba89ad-e4b9-43a6-b2c3-6904c93466bf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 55, 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:25.000165 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Network hostable DHCP agents. Network: 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:00:25.002758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Made reservation on behalf of c42fc1bb2ae6477fa7b7c9261f2d50bb for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:25.007090 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.029646 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Made reservation on behalf of 3f91ad6930924a8d81ee21ec1c490828 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:25.031919 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.041042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.058673 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.067613 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:25.074070 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:25.079347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:25.085739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:25.088611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:25.092817 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:25.097666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:25.100784 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:00:25.109449 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:25.178905 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:25.186801 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:25.186801 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.189262 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e16d5c3b-14fc-45ce-9f78-e87bebe246b5 None None] Network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:25.261349 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.276529 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.348240 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.349001 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 849 time: 1.0618949 May 24 10:00:25.355952 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:25.380772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 90} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:00:25.402911 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5779f0cc-6cbf-464c-be65-041f64dd4bc4,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:25.406194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:25.407555 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.408032 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Added segment f1abdab4-bf27-4888-b131-574a25f02d59 of type vlan for network c1fb75f5-deee-4236-9b04-1b6b5e617606 May 24 10:00:25.413590 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.420201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:25.428261 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d6be4712-3e29-4cc3-9cff-b99201c89d7f None None] Network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:25.481094 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.490078 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 61} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:00:25.492712 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=623bc222-7568-4ac6-9179-b8d061289a25,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:25.496801 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:25.508532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.508882 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Added segment 25064880-e56c-4a2f-b56c-21997837afc6 of type vlan for network e28486c2-462f-406b-8c64-3246ae76f56c May 24 10:00:25.564060 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.641600 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.647258 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:25.647894 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.658153 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f75df31-e558-4366-a72d-72b2a1e84c30 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:25.685429 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.701980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:25.702841 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:25.846577 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 1.3146341 May 24 10:00:25.846577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Pushing event updated for resources: {'Network': ['ID=c1fb75f5-deee-4236-9b04-1b6b5e617606,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:25.848419 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50756) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:25.855629 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Request body: {'subnet': {'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeTestJSON-127484096-subnet', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:25.858108 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:00:25.858514 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:25.859212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bebc6411-8627-4fa3-a321-099f91cb14b3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.142s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:25.869402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:25.877772 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Usage tracker for resource:subnet and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:25.894467 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:25.897840 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:25.901584 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 889 time: 1.6072130 May 24 10:00:25.907296 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50764) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:25.922326 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Request body: {'subnet': {'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-subnet', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:25.932295 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:25.939731 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Usage tracker for resource:subnet and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:25.943790 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:25.953960 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:25.987194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Pushing event updated for resources: {'Network': ['ID=e28486c2-462f-406b-8c64-3246ae76f56c,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:25.989796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Made reservation on behalf of 3f91ad6930924a8d81ee21ec1c490828 for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:25.990436 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:25.992132 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:26.056131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Made reservation on behalf of c42fc1bb2ae6477fa7b7c9261f2d50bb for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:26.060643 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:26.087765 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network e28486c2-462f-406b-8c64-3246ae76f56c details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:00:26.925917 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:26.933906 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-107006f8-8124-4836-89b2-92d29cb8a008 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:26.968687 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:26.983435 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50814) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:27.052800 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Checking segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:27.053188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Network hostable DHCP agents. Network: e28486c2-462f-406b-8c64-3246ae76f56c, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:00:27.059144 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Pushing event updated for resources: {'Subnet': ['ID=59e96feb-f5d8-4b1e-b08d-b93c34f5461c,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:27.064010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:27.064712 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.087s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:27.103914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Checking segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:27.104670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Network hostable DHCP agents. Network: c1fb75f5-deee-4236-9b04-1b6b5e617606, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:00:27.147298 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Pushing event updated for resources: {'Subnet': ['ID=327631ed-b242-499d-854c-7f681e3bd80e,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:27.151414 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:27.189989 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Network e28486c2-462f-406b-8c64-3246ae76f56c is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:00:27.213277 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:00:27.252617 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, e28486c2-462f-406b-8c64-3246ae76f56c) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:27.262017 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:27.263292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.263865 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d7c89d03-e246-4d12-a6f3-51aa4c2b4925 None None] Network e28486c2-462f-406b-8c64-3246ae76f56c requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:27.268685 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, c1fb75f5-deee-4236-9b04-1b6b5e617606) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:27.280128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:27.281033 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.291874 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-188ed88f-51b6-4f7f-b9e6-2752808f6cba None None] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:27.361564 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.5118530 May 24 10:00:27.364474 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50828) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:27.372237 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:27.381843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:27.390977 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Usage tracker for resource:router and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:27.395501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 853 time: 1.4868715 May 24 10:00:27.396591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:27.399310 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50832) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:27.404952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:27.408025 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:27.417372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:27.422981 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Usage tracker for resource:router and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:27.428171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:27.437344 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:27.502952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.505013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.549379 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.551209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.553955 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.580038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.684070 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:27.705454 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:27.710099 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Usage tracker for resource:security_group_rule and tenant:15bbb2d5d2fd454fa335b86a4c15a6cd is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:27.715284 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:27.723547 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:28.053097 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-c1fb75f5-deee-4236-9b04-1b6b5e617606', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:00:28.054155 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.140704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.167144 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-e28486c2-462f-406b-8c64-3246ae76f56c', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:00:28.170016 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.209010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.209568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50890) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:28.232894 dv-cinder-791239-2-iscsi neutron-server[109112]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 24 10:00:28.334579 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.474435 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.500857 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:28.538786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-c4be3b95-c011-46ad-a11a-9edf1294522c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:28.539912 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-c4be3b95-c011-46ad-a11a-9edf1294522c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Synchronizing usage tracker for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:28.555710 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Allocated IP 172.20.0.2 (e28486c2-462f-406b-8c64-3246ae76f56c/59e96feb-f5d8-4b1e-b08d-b93c34f5461c/e0e331ae-89b8-435b-bceb-b7531603a358) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:28.567839 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8b6c51fa-990e-4853-9a2f-a8421562f147,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:28.582129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:28.598507 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=15bbb2d5d2fd454fa335b86a4c15a6cd&name=default HTTP/1.1" status: 200 len: 2975 time: 1.6138673 May 24 10:00:28.601311 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 50908) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:28.701577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-c4be3b95-c011-46ad-a11a-9edf1294522c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Unset dirty status for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:28.702828 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-c4be3b95-c011-46ad-a11a-9edf1294522c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/networks?id=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 HTTP/1.1" status: 200 len: 812 time: 0.4921870 May 24 10:00:28.747963 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Allocated IP 172.20.0.2 (c1fb75f5-deee-4236-9b04-1b6b5e617606/327631ed-b242-499d-854c-7f681e3bd80e/3e6b3d69-2ada-4c22-8efb-d8aef48472ab) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:28.777635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.805444 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.882040 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.924566 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:28.925716 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-6d2e3690-caed-4157-80d7-f58ab5d5918a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/quotas/35a01011c3994805ba2a956d0daf4aa6 HTTP/1.1" status: 200 len: 375 time: 0.1588423 May 24 10:00:28.977639 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50918) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:28.993281 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.008800 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Transition to ACTIVE for port object 3e6b3d69-2ada-4c22-8efb-d8aef48472ab will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:29.014596 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-66a67aea-1c44-4c3d-9b55-d6bb8e813c72 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=35a01011c3994805ba2a956d0daf4aa6&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0878041 May 24 10:00:29.055276 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.091498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.129454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50926) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:29.175523 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Allocated IP 172.27.0.202 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/cd9a22d0-5e50-4294-9642-050e775c1046) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:29.186001 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.197178 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1baef4e3-1ab2-482b-9c73-53845c9420ec admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:29.200642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Transition to ACTIVE for port object 3e6b3d69-2ada-4c22-8efb-d8aef48472ab will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:29.219222 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8b6c51fa-990e-4853-9a2f-a8421562f147,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:29.226521 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:29.241476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.246172 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/8b6c51fa-990e-4853-9a2f-a8421562f147 HTTP/1.1" status: 204 len: 168 time: 0.6404541 May 24 10:00:29.286342 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.332578 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.332578 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Transition to ACTIVE for port object e0e331ae-89b8-435b-bceb-b7531603a358 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:29.361162 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Transition to ACTIVE for port object e0e331ae-89b8-435b-bceb-b7531603a358 will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:29.362674 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.367924 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:29.401336 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-a2d9ca11-1bc5-40ee-ad72-4d88caa06b14 service neutron] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109112) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:00:29.403969 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:29.409577 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-a2d9ca11-1bc5-40ee-ad72-4d88caa06b14 service neutron] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1048989 May 24 10:00:29.423792 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Usage tracker for resource:security_group_rule and tenant:29a032f9555344f389f69f2e1703a05d is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:29.429813 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:29.436343 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.439890 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:29.447048 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:29.451957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:29.455549 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:29.467125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:29.490487 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Usage tracker for resource:security_group_rule and tenant:c2e6772d2dca4e6f84220ccb04b7a9c6 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:29.502514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:29.505851 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.506652 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Pushing event updated for resources: {'Port': ['ID=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:29.528887 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:29.546259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.560622 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.560622 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.560622 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.561326 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.561326 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.561620 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.561943 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.562262 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.562609 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.562915 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.563233 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.563544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Tracked resource versions report: May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: pushed versions: May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: { 'AddressGroup': {'1.2'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'ConntrackHelper': {'1.0'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Log': {'1.0'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Network': {'1.1'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Port': {'1.7'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'PortForwarding': {'1.2'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'QosPolicy': {'1.8'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroup': {'1.4'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroupRule': {'1.2'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SubPort': {'1.0'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Subnet': {'1.1'}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Trunk': {'1.1'}} May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: consumer versions: May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: HyperV agent@hv-cinder-79123: {}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'ConntrackHelper': '1.0', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Log': '1.0', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Network': '1.1', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Port': '1.7', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'PortForwarding': '1.2', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'QosPolicy': '1.8', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroup': '1.4', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SecurityGroupRule': '1.2', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'SubPort': '1.0', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Subnet': '1.1', May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: 'Trunk': '1.1'}} May 24 10:00:29.564544 dv-cinder-791239-2-iscsi neutron-server[109114]: {{(pid=109114) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 10:00:29.591113 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.223s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:29.595837 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.044s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:29.633879 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-b1aba401-501a-4710-9ed1-2c23b7c26ca7 service neutron] 10.222.0.67 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2231734 May 24 10:00:29.665449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Network e28486c2-462f-406b-8c64-3246ae76f56c is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:29.670945 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:29.676186 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:29.681698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Attempting to bind port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:29.682079 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Attempting to bind port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:29.682443 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Attempting to bind port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:29.716317 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.759730 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.774655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Checking segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Bound using segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Bound port: 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:29.779070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.789618 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:29.793355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Pushing event updated for resources: {'Port': ['ID=e0e331ae-89b8-435b-bceb-b7531603a358,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:29.830993 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.235s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:29.846028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=712e32eb-5048-4963-9642-b1e3685f10d5,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:29.852844 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:29.858613 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.858613 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:29.917517 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-6e8cd640-afe0-4d4d-8835-9a1fc97f9714 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:29.920815 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Allocated IP 172.27.0.34 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/6f0c5b38-17fc-465c-926f-712944583a0a) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:29.924361 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-bb38335f-87a2-45fb-8972-9b4ccf30dc7d service neutron] 10.222.0.67 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2769327 May 24 10:00:29.927871 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Synchronizing usage tracker for tenant:ccff31096da8487cb3989249327c9e0c on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:29.970345 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Attempting to bind port e0e331ae-89b8-435b-bceb-b7531603a358 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:29.970744 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Attempting to bind port e0e331ae-89b8-435b-bceb-b7531603a358 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:29.971105 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Attempting to bind port e0e331ae-89b8-435b-bceb-b7531603a358 on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:29.983673 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Synchronizing usage tracker for tenant:ccff31096da8487cb3989249327c9e0c on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:30.007782 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=feabcc49-0fc8-45ea-927b-0503ab45b55b,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:30.021526 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:30.022736 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:30.023754 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.091s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:30.033603 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:30.033977 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Checking segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:30.034380 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.034776 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Bound using segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:30.035251 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Bound port: e0e331ae-89b8-435b-bceb-b7531603a358, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:30.035621 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.065860 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Unset dirty status for tenant:ccff31096da8487cb3989249327c9e0c on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:30.067312 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c2e6772d2dca4e6f84220ccb04b7a9c6&name=default HTTP/1.1" status: 200 len: 2975 time: 0.9369378 May 24 10:00:30.076578 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] For port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:30.078141 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50950) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:30.079405 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:30.083287 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:30.116591 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.136129 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] For port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:30.175182 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] For port e0e331ae-89b8-435b-bceb-b7531603a358, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:30.178682 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] For port e0e331ae-89b8-435b-bceb-b7531603a358, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e0e331ae-89b8-435b-bceb-b7531603a358,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:30.232883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.244789 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:30.252952 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.266060 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Unset dirty status for tenant:ccff31096da8487cb3989249327c9e0c on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:30.267198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=29a032f9555344f389f69f2e1703a05d&name=default HTTP/1.1" status: 200 len: 2975 time: 1.2875006 May 24 10:00:30.276142 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:30.293767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Pushing event updated for resources: {'Port': ['ID=cd9a22d0-5e50-4294-9642-050e775c1046,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:30.295756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50962) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:30.296089 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50964) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:30.296442 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 50966) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:30.380292 dv-cinder-791239-2-iscsi neutron-server[109113]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 24 10:00:30.395395 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.422376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.398s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:30.444589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.488854 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:30.494970 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:30.503873 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:30.527480 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2255ea96-5053-4364-be97-0c5fd11e6c06 req-aeede3e1-1533-4c66-9455-98de90564148 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.1923282 May 24 10:00:30.529990 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Pushing event updated for resources: {'Port': ['ID=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:30.540492 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.296s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:30.541564 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.065s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:30.601023 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:30.677929 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Usage tracker for resource:security_group_rule and tenant:469a454393bd49c08ebd0677c2cb14b7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:30.687187 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.689714 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Pushing event updated for resources: {'Port': ['ID=6f0c5b38-17fc-465c-926f-712944583a0a,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:30.727824 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:30.728018 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Pushing event updated for resources: {'Port': ['ID=e0e331ae-89b8-435b-bceb-b7531603a358,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:30.730763 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.740095 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.199s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:30.772289 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:30.777358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.512s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:30.852959 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:31.074163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.148037 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:31.198967 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.216021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=feabcc49-0fc8-45ea-927b-0503ab45b55b,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:31.242032 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:31.301900 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.481680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.482829 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/feabcc49-0fc8-45ea-927b-0503ab45b55b HTTP/1.1" status: 204 len: 168 time: 1.3403065 May 24 10:00:31.521389 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.625496 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.736792 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.755035 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.772843 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-fd28acdb-ffcb-49d9-81cf-893eac28685d req-9bc46283-e648-43dd-af58-3ca60e59f287 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.0864902 May 24 10:00:31.781531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f4c0d436-3e22-4382-ba75-5cd8a2a53e96 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:31.820739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:31.938776 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7bfceac9-419d-452a-a805-3c72134af203,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:31.954749 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e6766ee0-91a3-4624-a5d5-fcecc3f03bdd admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=469a454393bd49c08ebd0677c2cb14b7&name=default HTTP/1.1" status: 200 len: 2975 time: 1.6214325 May 24 10:00:31.959304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51030) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:32.208316 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=712e32eb-5048-4963-9642-b1e3685f10d5,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:32.260120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.268082 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/712e32eb-5048-4963-9642-b1e3685f10d5 HTTP/1.1" status: 204 len: 168 time: 1.9351885 May 24 10:00:32.279774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.493s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:32.316764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.468s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:32.321291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:32.352175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-913f9e5a-55da-44b1-82b0-aea36b814c1a None None] Network e28486c2-462f-406b-8c64-3246ae76f56c requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:32.394650 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Router cae69d10-56d3-4334-8870-67746df8a973 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:00:32.402883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.615441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.654228 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.670498 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e0e331ae-89b8-435b-bceb-b7531603a358 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:32.747756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51050) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:32.814080 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.823731 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-b5ca61dc-4788-4006-a6ea-b12c1bcaacf3 None None] Provisioning for port e0e331ae-89b8-435b-bceb-b7531603a358 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:32.855134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:32.858518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Usage tracker for resource:security_group_rule and tenant:da537a0aea64468396f7f21aa36d9318 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:32.863173 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:00:32.871413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:32.871803 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-73c09b78-b22f-432a-9da8-1848170258c8 None None] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:00:32.877656 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:00:32.887705 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:32.911603 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:32.913796 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:32.961304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d11f6e95-d67e-4eb9-a303-2d6da8e06c8b req-0ff69637-0572-47ef-876b-260da969f230 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.0700796 May 24 10:00:32.989565 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:32.990816 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 928 time: 5.5894547 May 24 10:00:32.999965 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51060) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:33.107967 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51064) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:33.122961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port e0e331ae-89b8-435b-bceb-b7531603a358 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:33.137020 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a4d47421-74f6-402f-a8c9-8e408814bf7c None None] Provisioning for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:33.158109 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port e0e331ae-89b8-435b-bceb-b7531603a358 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:33.159215 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.174183 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.208083 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:33.283554 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7bfceac9-419d-452a-a805-3c72134af203,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:33.293924 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:33.319413 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.319413 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/7bfceac9-419d-452a-a805-3c72134af203 HTTP/1.1" status: 204 len: 168 time: 1.3556864 May 24 10:00:33.323631 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2aad6d37-4e29-4d5e-b490-f4394c9c5cb3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.417588 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:33.448355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.480110 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Router 1ec02266-2164-4194-a981-4dcbb8771c01 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109113) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:00:33.509396 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port e0e331ae-89b8-435b-bceb-b7531603a358, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e0e331ae-89b8-435b-bceb-b7531603a358,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:33.595246 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.597168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 3e6b3d69-2ada-4c22-8efb-d8aef48472ab up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:33.603549 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.677895 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:33.762673 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.780454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:33.781668 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:33.843512 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=94b56cb1-8b84-4dea-959a-3c3941feebd2,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:33.865408 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:33.870494 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Usage tracker for resource:security_group_rule and tenant:62c0b6c6cbf540919fed9fdec2df173d is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:33.872398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:33.874288 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:33.881361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:33.898333 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=da537a0aea64468396f7f21aa36d9318&name=default HTTP/1.1" status: 200 len: 2975 time: 1.1441414 May 24 10:00:33.899113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51082) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:33.975280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-f6008742-faeb-4ec0-8c67-1ec6caac0eb6 None None] Provisioning complete for port e0e331ae-89b8-435b-bceb-b7531603a358 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:33.975984 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-f6008742-faeb-4ec0-8c67-1ec6caac0eb6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.016794 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.089098 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.124854 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-74ee5fbd-e6e7-4f6b-b642-8a790db621ad admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:34.156386 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:29Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:34.171236 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:00:34.171395 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Updating router cae69d10-56d3-4334-8870-67746df8a973 port 6f0c5b38-17fc-465c-926f-712944583a0a binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:00:34.201203 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:00:34.206582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=100f0345-3555-4ffa-a27d-b32cf1fac882,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:34.224893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=e0e331ae-89b8-435b-bceb-b7531603a358,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:34.240848 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:00:34.250445 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:34.253890 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=62c0b6c6cbf540919fed9fdec2df173d&name=default HTTP/1.1" status: 200 len: 2975 time: 1.1443138 May 24 10:00:34.258605 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51100) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:34.263462 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.586s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:34.272134 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:34.273729 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.354178 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.227s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:34.360301 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.458803 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.460346 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:34.461004 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:34.464187 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.469041 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 7.1006145 May 24 10:00:34.481020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51108) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:34.491559 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9b4cb38-d9b9-4d76-b35f-543e7acc0166 req-4f0207a9-9da2-4f6c-a01e-6889ef61a91c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.1528420 May 24 10:00:34.639465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.668366 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.699104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.868535 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.877501 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5f62abe4-5585-4305-8c39-6960e2d25cc2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:34.901392 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=94b56cb1-8b84-4dea-959a-3c3941feebd2,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:34.913268 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:34.929675 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:34.929675 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/94b56cb1-8b84-4dea-959a-3c3941feebd2 HTTP/1.1" status: 204 len: 168 time: 1.0044994 May 24 10:00:34.944859 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.111779 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.141707 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.163436 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Transition to ACTIVE for port object 6f0c5b38-17fc-465c-926f-712944583a0a will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:35.186939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:35.204576 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.95', 58118) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:35.253587 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.268756 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:35.377773 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:35.444669 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-602bdf8d-c369-44c3-845f-2111e4917316 None None] Provisioning complete for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:35.445285 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-602bdf8d-c369-44c3-845f-2111e4917316 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.448913 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Pushing event updated for resources: {'Port': ['ID=6f0c5b38-17fc-465c-926f-712944583a0a,revision_number=2']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:35.464214 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.464828 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.465349 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.466049 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.466708 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.495594 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.496275 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.497338 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.498227 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.500826 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.503041 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.504036 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.504893 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Tracked resource versions report: May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: pushed versions: May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: { 'AddressGroup': {'1.2'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'ConntrackHelper': {'1.0'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Log': {'1.0'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Network': {'1.1'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Port': {'1.7'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'PortForwarding': {'1.2'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'QosPolicy': {'1.8'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroup': {'1.4'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroupRule': {'1.2'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SubPort': {'1.0'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Subnet': {'1.1'}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Trunk': {'1.1'}} May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: consumer versions: May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: { DHCP agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: HyperV agent@hv-cinder-79123: {}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: L3 agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: Metadata agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: Metering agent@dv-cinder-791239-2-iscsi: {}, May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: Open vSwitch agent@dv-cinder-791239-2-iscsi: { 'AddressGroup': '1.2', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'ConntrackHelper': '1.0', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Log': '1.0', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Network': '1.1', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Port': '1.7', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'PortForwarding': '1.2', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'QosPolicy': '1.8', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroup': '1.4', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SecurityGroupRule': '1.2', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'SubPort': '1.0', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Subnet': '1.1', May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: 'Trunk': '1.1'}} May 24 10:00:35.506725 dv-cinder-791239-2-iscsi neutron-server[109115]: {{(pid=109115) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 10:00:35.519872 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Allocated IP 172.20.0.1 (c1fb75f5-deee-4236-9b04-1b6b5e617606/327631ed-b242-499d-854c-7f681e3bd80e/367e9138-2d81-4a46-8056-5d07308e918d) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:35.521584 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-834d9fd0-e313-4171-873d-11c7320f8bf2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:35.571603 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.303s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:35.617461 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.648181 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Attempting to bind port 6f0c5b38-17fc-465c-926f-712944583a0a on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:35.648578 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Attempting to bind port 6f0c5b38-17fc-465c-926f-712944583a0a by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:35.648941 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Attempting to bind port 6f0c5b38-17fc-465c-926f-712944583a0a on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:35.660631 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d2255ad7-e549-40b7-9a31-a20d5f8c5844 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:35.663489 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:35.664650 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:35.664650 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.664650 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:35.665010 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Bound port: 6f0c5b38-17fc-465c-926f-712944583a0a, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:35.665461 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.743205 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e0e331ae-89b8-435b-bceb-b7531603a358 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:35.805219 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] For port 6f0c5b38-17fc-465c-926f-712944583a0a, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:35.810210 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] For port 6f0c5b38-17fc-465c-926f-712944583a0a, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=6f0c5b38-17fc-465c-926f-712944583a0a,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:35.871835 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:35.876060 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.881079 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.359s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:35.884275 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7191f517-f908-419f-8902-d17577593737 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:35.894021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=100f0345-3555-4ffa-a27d-b32cf1fac882,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:35.903047 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.250s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:35.950593 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.955040 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/100f0345-3555-4ffa-a27d-b32cf1fac882 HTTP/1.1" status: 204 len: 168 time: 1.6900465 May 24 10:00:35.981871 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3e0f5dea-e0a9-419d-9888-75a792a4c925 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.984060 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:35.984782 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Pushing event updated for resources: {'Port': ['ID=6f0c5b38-17fc-465c-926f-712944583a0a,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:35.993291 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:36.013534 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:36.015426 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:36.100441 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.101106 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port e0e331ae-89b8-435b-bceb-b7531603a358 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:36.101430 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.110442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-dd3eff04-530d-40ed-8cda-116f5ac7a50c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:36.111933 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-dd3eff04-530d-40ed-8cda-116f5ac7a50c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.95 "GET /v2.0/networks?id=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 HTTP/1.1" status: 200 len: 812 time: 0.9063478 May 24 10:00:36.138190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Transition to ACTIVE for port object 367e9138-2d81-4a46-8056-5d07308e918d will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:36.141945 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:29Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:36.167274 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:00:36.167641 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Updating router 1ec02266-2164-4194-a981-4dcbb8771c01 port cd9a22d0-5e50-4294-9642-050e775c1046 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:00:36.235823 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.246666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.252219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:36.299907 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:36.308430 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cb77dfc7-e65e-48ac-a02f-f0675605be25 req-62e7dcfc-4bcf-4331-ac81-ee5911f05d27 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.3456352 May 24 10:00:36.350096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:36.358684 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:36.373773 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:36.374542 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.571957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.598201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Pushing event updated for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:36.605451 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.353s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:36.721097 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.859953 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:36.868039 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21784360-7cc9-40d3-b6fd-0701bb802998 None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:00:36.870049 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e3cd1531-acd0-4580-9cd2-615b95a9c841 None None] Provisioning for port 367e9138-2d81-4a46-8056-5d07308e918d completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:36.896483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e3cd1531-acd0-4580-9cd2-615b95a9c841 None None] Provisioning complete for port 367e9138-2d81-4a46-8056-5d07308e918d triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:36.896483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e3cd1531-acd0-4580-9cd2-615b95a9c841 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:36.962363 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Allocated IP 172.20.0.1 (e28486c2-462f-406b-8c64-3246ae76f56c/59e96feb-f5d8-4b1e-b08d-b93c34f5461c/ed75d389-b3db-45f5-bf82-87fe5209a3e9) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:36.998193 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e3cd1531-acd0-4580-9cd2-615b95a9c841 None None] Port 367e9138-2d81-4a46-8056-5d07308e918d cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:00:37.032305 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.039002 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.060653 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Transition to ACTIVE for port object cd9a22d0-5e50-4294-9642-050e775c1046 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:37.120269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.121622 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.126325 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-dfad5665-b721-4baf-af42-a62971d16ef2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:37.178037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.184327 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:37.195612 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.218409 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Transition to ACTIVE for port object ed75d389-b3db-45f5-bf82-87fe5209a3e9 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:37.296469 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.356365 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Attempting to bind port cd9a22d0-5e50-4294-9642-050e775c1046 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:37.357161 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Attempting to bind port cd9a22d0-5e50-4294-9642-050e775c1046 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:37.357750 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Attempting to bind port cd9a22d0-5e50-4294-9642-050e775c1046 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:37.361913 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.368163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d42eb851-59b5-41bd-94b4-13399eea7de1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:37.373910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Pushing event updated for resources: {'Port': ['ID=cd9a22d0-5e50-4294-9642-050e775c1046,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:37.377288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:37.380565 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:37.381253 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:37.381886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.382832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:37.383623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Bound port: cd9a22d0-5e50-4294-9642-050e775c1046, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:37.384149 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.436251 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51190) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:37.456575 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:37.462583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Pushing event updated for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:37.471501 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.345s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:37.474961 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:37.506850 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.511050 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] For port cd9a22d0-5e50-4294-9642-050e775c1046, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:37.520352 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] For port cd9a22d0-5e50-4294-9642-050e775c1046, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=cd9a22d0-5e50-4294-9642-050e775c1046,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:37.553309 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:37.559654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Usage tracker for resource:security_group_rule and tenant:8d8fbfbef68f43b996aa8c6784c8b806 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:37.561399 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:37.567970 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:37.583722 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.592149 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:37.701527 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.706720 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Network e28486c2-462f-406b-8c64-3246ae76f56c is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:37.710591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:37.714405 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:37.715216 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.870838 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 6f0c5b38-17fc-465c-926f-712944583a0a up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:37.934008 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:37.934856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Pushing event updated for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:37.991935 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a4c464d8-4d09-441a-86ec-a488a301f692 req-7398a02f-d797-4569-9749-c9889bd488ea tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.0937240 May 24 10:00:37.992351 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.624s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:37.992904 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:38.060821 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:00:38.092900 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-8b8ab134-74ac-4d05-99f6-795f8eb0c339 None None] Provisioning complete for port 367e9138-2d81-4a46-8056-5d07308e918d triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:38.093445 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8b8ab134-74ac-4d05-99f6-795f8eb0c339 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.347786 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:38.347786 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:38.349766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3394182f-03f0-4734-9ab0-7ee1bf0c475d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:38.371006 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.353s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:38.411687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51214) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:38.412886 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=8d8fbfbef68f43b996aa8c6784c8b806&name=default HTTP/1.1" status: 200 len: 2975 time: 0.9397020 May 24 10:00:38.433961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Pushing event updated for resources: {'Port': ['ID=cd9a22d0-5e50-4294-9642-050e775c1046,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:38.478427 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.871s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:38.480233 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Request body: {'port': {'device_id': 'af25f39c-cd76-47bd-aace-363ee0770877', 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138', 'admin_state_up': True, 'tenant_id': '35a01011c3994805ba2a956d0daf4aa6'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:38.499095 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:00:38.499731 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:38.610523 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] 10.222.0.67 "PUT /v2.0/routers/cae69d10-56d3-4334-8870-67746df8a973/add_router_interface HTTP/1.1" status: 200 len: 503 time: 5.6001005 May 24 10:00:38.642843 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.690399 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-8b8ab134-74ac-4d05-99f6-795f8eb0c339 None None] Port 367e9138-2d81-4a46-8056-5d07308e918d cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:00:38.691897 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 6f0c5b38-17fc-465c-926f-712944583a0a completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:38.700161 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.701086 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.710203 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:38.725579 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Usage tracker for resource:port and tenant:35a01011c3994805ba2a956d0daf4aa6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:38.740096 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 6f0c5b38-17fc-465c-926f-712944583a0a triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:38.740552 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Quota usage for port was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:38.740752 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.759027 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:38.898988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Made reservation on behalf of 35a01011c3994805ba2a956d0daf4aa6 for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:38.905547 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.910804 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:38.960892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.009952 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ae2c19ba-f490-4cd2-a9e3-dc01e12c45f9 None None] Provisioning for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:39.057527 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ae2c19ba-f490-4cd2-a9e3-dc01e12c45f9 None None] Provisioning complete for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:39.057913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ae2c19ba-f490-4cd2-a9e3-dc01e12c45f9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.066482 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:39.100012 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3394182f-03f0-4734-9ab0-7ee1bf0c475d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:39.124599 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:39.164078 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.204712 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-ae2c19ba-f490-4cd2-a9e3-dc01e12c45f9 None None] Port ed75d389-b3db-45f5-bf82-87fe5209a3e9 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:00:39.210852 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 6f0c5b38-17fc-465c-926f-712944583a0a, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=6f0c5b38-17fc-465c-926f-712944583a0a,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:39.255275 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:37Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:39.268269 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4fb3a9b3-c750-4121-8cff-8a5b21ff33ed None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:30Z', 'revision_number': 3, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:37Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:00:39.268520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.269842 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/3394182f-03f0-4734-9ab0-7ee1bf0c475d HTTP/1.1" status: 204 len: 168 time: 0.8566830 May 24 10:00:39.272450 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-83a6bd46-c2f4-4501-9325-a5c62ac25189 req-511ff5e0-8287-49ef-8243-4fb212712c44 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.0939939 May 24 10:00:39.287551 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.288740 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 3e6b3d69-2ada-4c22-8efb-d8aef48472ab up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:39.334869 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.348214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:39.433681 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Allocated IP 172.20.0.9 (1c6bd2e7-cf1c-4cda-aa53-7f057dec1138/b28ebd86-e35c-4b85-b6ba-3336770f64e8/35e9079c-497b-4448-a787-0fdee4f4fc4c) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:39.456895 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.463402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.463867 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:39.485626 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:00:39.505879 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:39.506512 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:39.530047 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=6f0c5b38-17fc-465c-926f-712944583a0a,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:39.541730 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.194s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:39.564441 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:39.565166 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:39.901144 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:39.987580 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Pushing event updated for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:39.998938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.535s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:40.146715 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Network e28486c2-462f-406b-8c64-3246ae76f56c is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:40.151660 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:40.172663 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:40.304130 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:40.338672 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-38fe3d35-2bdf-4e74-a7ed-10004abcdd31 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:40.339683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 6f0c5b38-17fc-465c-926f-712944583a0a up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:40.538481 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:40.548461 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:40.573029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:40.595039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 6f0c5b38-17fc-465c-926f-712944583a0a triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:40.595484 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:40.605523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:40.606168 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:40.678045 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Pushing event updated for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:40.745024 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-09574465-2274-4e06-a84f-fd479f6d1b94 None None] Provisioning complete for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:40.745878 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-09574465-2274-4e06-a84f-fd479f6d1b94 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:40.807073 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:00:40.834358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.494s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:40.927573 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-09574465-2274-4e06-a84f-fd479f6d1b94 None None] Port ed75d389-b3db-45f5-bf82-87fe5209a3e9 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:00:40.957040 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device cd9a22d0-5e50-4294-9642-050e775c1046 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:40.972617 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 2, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:40.985584 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:00:40.985946 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Updating router cae69d10-56d3-4334-8870-67746df8a973 port 367e9138-2d81-4a46-8056-5d07308e918d binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:00:41.011296 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.5511327 May 24 10:00:41.022423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-709a4675-2c1d-4121-8a8e-23c5e9e42c22 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109112) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:00:41.030307 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-709a4675-2c1d-4121-8a8e-23c5e9e42c22 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.95 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0120668 May 24 10:00:41.057230 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8010727d-871f-4b3b-96bf-07737bb69195 req-ea46ade5-ed1a-4e18-9fcf-46406625de94 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:41.059617 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8010727d-871f-4b3b-96bf-07737bb69195 req-ea46ade5-ed1a-4e18-9fcf-46406625de94 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Synchronizing usage tracker for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:41.105208 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.132303 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port cd9a22d0-5e50-4294-9642-050e775c1046 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:41.139303 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port cd9a22d0-5e50-4294-9642-050e775c1046 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:41.140068 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.183748 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8010727d-871f-4b3b-96bf-07737bb69195 req-ea46ade5-ed1a-4e18-9fcf-46406625de94 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Unset dirty status for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:41.185076 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8010727d-871f-4b3b-96bf-07737bb69195 req-ea46ade5-ed1a-4e18-9fcf-46406625de94 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 931 time: 0.5343523 May 24 10:00:41.268714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:00:41.278793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8010727d-871f-4b3b-96bf-07737bb69195 req-e5c01c45-6f9f-407e-9960-21eff23346bd tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Synchronizing usage tracker for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:41.319068 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] 10.222.0.67 "PUT /v2.0/routers/1ec02266-2164-4194-a981-4dcbb8771c01/add_router_interface HTTP/1.1" status: 200 len: 503 time: 6.8294339 May 24 10:00:41.360701 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8010727d-871f-4b3b-96bf-07737bb69195 req-e5c01c45-6f9f-407e-9960-21eff23346bd tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Unset dirty status for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:41.361998 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8010727d-871f-4b3b-96bf-07737bb69195 req-e5c01c45-6f9f-407e-9960-21eff23346bd tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1707489 May 24 10:00:41.377081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51312) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:41.384043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1492441025', 'description': 'tempest-description--885501607'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:41.396858 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:41.407818 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:41.412475 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:41.429540 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:41.542713 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.582573 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port cd9a22d0-5e50-4294-9642-050e775c1046, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=cd9a22d0-5e50-4294-9642-050e775c1046,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:41.584170 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.684397 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:41.699329 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.716082 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:41.732506 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:41.744345 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:41.757789 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:41.892413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51332) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:41.971346 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=cd9a22d0-5e50-4294-9642-050e775c1046,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:41.975291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:41.987060 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:42.072056 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.097980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Request body: {'port': {'device_id': 'af25f39c-cd76-47bd-aace-363ee0770877', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:42.167344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0b98c457-f2e6-408c-82a9-c196d58b8acd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:42.182652 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device cd9a22d0-5e50-4294-9642-050e775c1046 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:42.195580 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.236065 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Transition to ACTIVE for port object 367e9138-2d81-4a46-8056-5d07308e918d will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:42.257100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.264673 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.8816257 May 24 10:00:42.281623 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51350) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:42.308163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group_rule': {'security_group_id': 'd4764188-82ae-444d-ae05-a4e09e8d56dd', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:42.315064 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.322302 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:42.329995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:42.342100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:42.354613 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:42.358218 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.373571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:42.409897 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroup': ['ID=d4764188-82ae-444d-ae05-a4e09e8d56dd,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:42.425532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:42.430750 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:00:42.431101 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.263s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:42.438371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Usage tracker for resource:security_group_rule and tenant:7d18722ed6b34508a2ff4cbbf4393cbb is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:42.455259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:42.477150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.480973 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:42.510547 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:42.511323 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:42.652269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.656648 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.662270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.755469 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.766030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.796295 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-16e8b617-5123-4ebe-a31c-de8158d462c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:42.856700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:42.896680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=e095467f-e226-4cea-b408-35099b968dfa,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:42.913529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:42.915271 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.917764 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.6315002 May 24 10:00:42.927521 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51362) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:42.939998 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port cd9a22d0-5e50-4294-9642-050e775c1046 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:42.943033 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group_rule': {'security_group_id': 'd4764188-82ae-444d-ae05-a4e09e8d56dd', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:42.948192 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:42.960558 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:42.971764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:42.982863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:42.997634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:43.133240 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.146468 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Pushing event updated for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:43.169815 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.845s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:43.236840 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Attempting to bind port 367e9138-2d81-4a46-8056-5d07308e918d on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:43.237929 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Attempting to bind port 367e9138-2d81-4a46-8056-5d07308e918d by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:43.242873 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Attempting to bind port 367e9138-2d81-4a46-8056-5d07308e918d on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:43.252947 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:43.256703 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Checking segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:43.257367 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.257923 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Bound using segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:43.258567 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Bound port: 367e9138-2d81-4a46-8056-5d07308e918d, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:43.259139 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.306309 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ddf278d5-3ac8-4423-ab1d-194c056835ba,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:43.326956 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.531s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:43.369835 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=7d18722ed6b34508a2ff4cbbf4393cbb&name=default HTTP/1.1" status: 200 len: 2979 time: 1.4755561 May 24 10:00:43.372796 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51370) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:43.450638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.521221 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.524478 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:43.582459 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] For port 367e9138-2d81-4a46-8056-5d07308e918d, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:43.594907 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] For port 367e9138-2d81-4a46-8056-5d07308e918d, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=367e9138-2d81-4a46-8056-5d07308e918d,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:43.675765 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.688724 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:43.713774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=1d2370ad-de29-48ae-bab6-5d22f925eb15,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:43.719881 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.195s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:43.725392 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:43.791478 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Pushing event updated for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=4']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:43.797835 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:43.811156 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:43.811793 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:43.833436 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:43.833855 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.9026542 May 24 10:00:43.836777 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51384) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:43.844323 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:43.885589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:43.904868 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:floatingip and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:43.938517 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:43.960384 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:44.014014 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51390) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:44.019288 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1286635211', 'description': 'tempest-description--1135024532'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:44.033940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:44.054547 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:44.065096 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:44.097353 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:44.098575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.126478 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:44.177759 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.232523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-834e006d-af2f-4cb3-a498-021ac51afd08 req-8bd03e86-cae9-4b74-ba8b-19e9ea30a879 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:44.251510 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-834e006d-af2f-4cb3-a498-021ac51afd08 req-8bd03e86-cae9-4b74-ba8b-19e9ea30a879 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 931 time: 1.3002088 May 24 10:00:44.364740 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.433225 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.450842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:44.481772 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:44.497596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:44.507538 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:44.534961 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-834e006d-af2f-4cb3-a498-021ac51afd08 req-f32551f9-3584-4a37-bb9b-cc5f2991a590 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2799354 May 24 10:00:44.591912 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51398) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:44.672086 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.812793 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:00:44.871547 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:43Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:44.877859 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 35e9079c-497b-4448-a787-0fdee4f4fc4c {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:00:44.882210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 2, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:44.887178 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d78c008e-01bf-4bf9-b318-bd944981c95e None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:43Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:00:44.892973 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:44.895132 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:00:44.895743 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Updating router 1ec02266-2164-4194-a981-4dcbb8771c01 port ed75d389-b3db-45f5-bf82-87fe5209a3e9 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:00:45.008686 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.011923 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.039690 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Transition to ACTIVE for port object 35e9079c-497b-4448-a787-0fdee4f4fc4c will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:45.040793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.052187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-27e10349-9c45-4363-98a1-43849d967385 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:45.074991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ddf278d5-3ac8-4423-ab1d-194c056835ba,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:45.086129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:45.089556 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.090791 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/ddf278d5-3ac8-4423-ab1d-194c056835ba HTTP/1.1" status: 204 len: 168 time: 1.6994133 May 24 10:00:45.121139 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.126719 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3513dba6-5d32-4525-9d19-04d4356227b1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:45.190852 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.241480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.280268 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4a4e1f4-ea3a-47d4-8fc0-f51ecdc0ae84 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:45.347973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:45.439405 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Usage tracker for resource:security_group_rule and tenant:46a970c059a74969a4c9872dd4d573e8 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:45.447440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:45.454662 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroup': ['ID=c864d029-e730-4251-9cfd-084868a9275d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:45.457856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.462555 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1780 time: 1.4474773 May 24 10:00:45.470148 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51424) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:45.482237 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.202s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:45.493220 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:45.507096 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group_rule': {'security_group_id': 'c864d029-e730-4251-9cfd-084868a9275d', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:45.531072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Pushing event updated for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:45.535422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:45.553736 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.422s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:45.553822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:45.555186 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:45.586587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Allocated IP 172.27.0.38 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/43451a90-88e8-4a57-b11e-8d47a160d020) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:45.589738 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:45.772122 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Attempting to bind port 35e9079c-497b-4448-a787-0fdee4f4fc4c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:45.772643 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Attempting to bind port 35e9079c-497b-4448-a787-0fdee4f4fc4c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'deba89ad-e4b9-43a6-b2c3-6904c93466bf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 55, 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:45.775134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Attempting to bind port 35e9079c-497b-4448-a787-0fdee4f4fc4c on network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:45.790679 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Port 35e9079c-497b-4448-a787-0fdee4f4fc4c on network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:00:45.795456 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Attempting to bind port 35e9079c-497b-4448-a787-0fdee4f4fc4c on network 1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:45.808014 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:45.808516 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Checking segment: {'id': 'deba89ad-e4b9-43a6-b2c3-6904c93466bf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 55, 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:45.809187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Bound using segment: {'id': 'deba89ad-e4b9-43a6-b2c3-6904c93466bf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 55, 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:45.809710 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Bound port: 35e9079c-497b-4448-a787-0fdee4f4fc4c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'deba89ad-e4b9-43a6-b2c3-6904c93466bf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 55, 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:45.813037 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:45.908121 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.048078 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:46.053395 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c,segment=NetworkSegment(deba89ad-e4b9-43a6-b2c3-6904c93466bf),segment_id=deba89ad-e4b9-43a6-b2c3-6904c93466bf)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:46.121513 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.145734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Transition to ACTIVE for port object ed75d389-b3db-45f5-bf82-87fe5209a3e9 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:46.195133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.205202 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:46.217874 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.224994 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:46.351932 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 367e9138-2d81-4a46-8056-5d07308e918d up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:46.435747 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Pushing event updated for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:46.451510 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.246s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:46.493128 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:46.495418 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.498471 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] 10.222.0.95 "PUT /v2.0/ports/35e9079c-497b-4448-a787-0fdee4f4fc4c HTTP/1.1" status: 200 len: 1093 time: 4.7590017 May 24 10:00:46.500546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:00:46.642192 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51452) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:46.754421 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.924705 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:46.943973 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-b2b63fe9-20e9-45e5-878a-0d50676ca75c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:46.952226 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-b2b63fe9-20e9-45e5-878a-0d50676ca75c service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=35a01011c3994805ba2a956d0daf4aa6&device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 1096 time: 0.2563536 May 24 10:00:46.985902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.035918 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.044714 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Pushing event updated for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:47.076790 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.100534 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51468) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:47.101514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f104f923-0c73-4e7b-9670-7f9b1a5b0f65 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:47.148835 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.924s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:47.159911 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-77667038-898a-4024-91c5-38c92f3d80b7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c HTTP/1.1" status: 200 len: 217 time: 0.1625156 May 24 10:00:47.166795 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:47.227369 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 367e9138-2d81-4a46-8056-5d07308e918d completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:47.242195 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=46a970c059a74969a4c9872dd4d573e8&name=default HTTP/1.1" status: 200 len: 2979 time: 2.6486151 May 24 10:00:47.244607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Usage tracker for resource:security_group_rule and tenant:bcced7acc3724ef8a71b2535a8b62092 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:47.261691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0988794c-a352-46e2-95b9-7006fa48ad33,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:47.264880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-1e782115-bde8-45a4-838e-3b17663b61a7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:47.274340 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.164s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:47.275178 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-1e782115-bde8-45a4-838e-3b17663b61a7 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=b28ebd86-e35c-4b85-b6ba-3336770f64e8 HTTP/1.1" status: 200 len: 852 time: 0.0841300 May 24 10:00:47.275473 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:47.275763 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51480) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:47.275909 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.279422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-73650df4-20c6-41ab-9aeb-5e91b2c040b4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.100s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:47.284319 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 367e9138-2d81-4a46-8056-5d07308e918d triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:47.288554 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.370462 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Attempting to bind port ed75d389-b3db-45f5-bf82-87fe5209a3e9 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:00:47.373372 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Attempting to bind port ed75d389-b3db-45f5-bf82-87fe5209a3e9 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:00:47.377613 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Attempting to bind port ed75d389-b3db-45f5-bf82-87fe5209a3e9 on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:00:47.394758 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=43451a90-88e8-4a57-b11e-8d47a160d020,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:47.395653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:47.411356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:47.413088 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:47.427897 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f8115f3-8925-4c94-9fb4-d3890544df0d req-a2ec650c-f1ca-4b23-be1f-403956ff5454 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:47.433537 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8f8115f3-8925-4c94-9fb4-d3890544df0d req-a2ec650c-f1ca-4b23-be1f-403956ff5454 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 1.6787484 May 24 10:00:47.435186 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:47.446195 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:47.460905 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:00:47.460905 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Checking segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:00:47.461698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.462301 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Bound using segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:00:47.463026 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Bound port: ed75d389-b3db-45f5-bf82-87fe5209a3e9, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:00:47.463612 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.587174 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c', 'status': 'completed', 'code': 200} May 24 10:00:47.589944 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.639298 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.705500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-803a6a62-839a-459c-8979-23907e7994b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:47.726836 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=2d15e759-f956-45d9-a551-471067f62517,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:47.729793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.734822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:47.734822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:47.764844 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Usage tracker for resource:security_group_rule and tenant:10edebad2bfd4d6c8f593668abe7b919 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:47.773738 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.799605 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8f8115f3-8925-4c94-9fb4-d3890544df0d req-520d7d5c-5f3b-41b9-a2b4-1358ab1e1d14 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1756294 May 24 10:00:47.800570 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:47.813748 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:47.816347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.842290 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.851368 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 2.3585439 May 24 10:00:47.851368 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51492) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:47.855354 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-f3d1ef40-647c-4f99-a72d-38e23f852383 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.2672515 May 24 10:00:47.861021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group_rule': {'security_group_id': 'c864d029-e730-4251-9cfd-084868a9275d', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:47.869509 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:47.874112 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:47.879341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:47.879518 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] For port ed75d389-b3db-45f5-bf82-87fe5209a3e9, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:00:47.884206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:47.886767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:47.903027 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] For port ed75d389-b3db-45f5-bf82-87fe5209a3e9, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ed75d389-b3db-45f5-bf82-87fe5209a3e9,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:00:47.971698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 367e9138-2d81-4a46-8056-5d07308e918d, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=367e9138-2d81-4a46-8056-5d07308e918d,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:48.067642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.070665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.080096 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.085754 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-bdef259d-57f7-49ee-a6b6-8ddda436234a service neutron] 10.222.0.95 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2033892 May 24 10:00:48.117075 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.164804 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Pushing event updated for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.178503 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.189544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.190244 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.203631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.209167 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.212565 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.232397 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.234842 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/0988794c-a352-46e2-95b9-7006fa48ad33 HTTP/1.1" status: 204 len: 168 time: 0.9660363 May 24 10:00:48.236406 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0988794c-a352-46e2-95b9-7006fa48ad33,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.240964 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.241955 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.312104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.322032 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.364491 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:00:48.365187 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:00:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:48.377453 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.383701 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.426523 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.434008 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=bcced7acc3724ef8a71b2535a8b62092&name=default HTTP/1.1" status: 200 len: 2979 time: 1.4994748 May 24 10:00:48.434758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51502) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:48.553663 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.563323 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.373s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.590446 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-238e194d-06cd-4f8a-bab3-b4b9cf07fe22 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e4f116b4-0b6e-4313-a91b-825b948a8e9f,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.609605 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=10edebad2bfd4d6c8f593668abe7b919&name=default HTTP/1.1" status: 200 len: 2979 time: 1.4544506 May 24 10:00:48.614061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51512) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:48.617734 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.779243 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:00:48.789487 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c5c0c7f1-a934-431e-a151-9647fd7bc512,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.834107 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.450s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.834587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.409s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.834886 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.843937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-38950c41-03e2-4777-91f0-a844278fb29e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.375s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:48.881027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ea376633-5edd-4659-a3dc-0289654f7022,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:48.887213 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:48.935003 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:48.938617 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 1.0822990 May 24 10:00:48.983610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51524) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:49.000566 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.061553 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.062418 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.067600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:49.077776 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:49.093084 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:49.097287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:floatingip and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:49.100311 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:49.107018 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:49.169540 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.196672 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=43451a90-88e8-4a57-b11e-8d47a160d020,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:49.198558 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:49.270762 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:00:49.275046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40f69bda-f791-451a-af4a-fed51d62902f None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:00:49.299048 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.317084 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 5.4625566 May 24 10:00:49.317289 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.325275 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.457783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.485190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51540) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:49.527777 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a325a623-b477-4cbc-8c54-cb0534789d26 req-2a7bf075-9a0e-4972-ad97-eb368eac7236 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:49.537628 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a325a623-b477-4cbc-8c54-cb0534789d26 req-2a7bf075-9a0e-4972-ad97-eb368eac7236 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.4486849 May 24 10:00:49.551924 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.561907 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-67bf23ab-16ef-4e1c-bcf5-35ea33bd5abd req-4dfad7c3-f9ab-4495-b1ac-e9872c494e81 service neutron] 10.222.0.95 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.8104413 May 24 10:00:49.593032 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:49.600838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Usage tracker for resource:security_group_rule and tenant:2ee76931177b477f8c6eafa067b78c15 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:49.606013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:49.613817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:49.737795 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a325a623-b477-4cbc-8c54-cb0534789d26 req-87455b50-8dae-42a1-9af9-bad6ec5f0e7e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1991754 May 24 10:00:49.742746 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.751124 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3235a699-9654-4d0e-b133-1ccd929deb25 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:49.833002 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e4f116b4-0b6e-4313-a91b-825b948a8e9f,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:49.841098 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:49.873893 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51550) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:49.883133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.893110 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.899051 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/c5c0c7f1-a934-431e-a151-9647fd7bc512 HTTP/1.1" status: 204 len: 168 time: 1.2815099 May 24 10:00:49.899051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:49.899051 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e4f116b4-0b6e-4313-a91b-825b948a8e9f HTTP/1.1" status: 204 len: 168 time: 1.4626896 May 24 10:00:49.903231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:49.917333 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c5c0c7f1-a934-431e-a151-9647fd7bc512,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:49.942132 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:50.095643 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-2c52a805-9277-4a1a-9cc9-7e9b94c2bca6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:50.098213 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-2c52a805-9277-4a1a-9cc9-7e9b94c2bca6 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=35a01011c3994805ba2a956d0daf4aa6&device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 1096 time: 0.3281472 May 24 10:00:50.115578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:50.161266 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:50.164809 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-a3fab0b9-d729-4909-a56d-b6bd8b57023b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:50.297440 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:50.425651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:50.489532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:50.571157 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Usage tracker for resource:security_group_rule and tenant:efa6f3438def4305baf64679ad31ac78 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:50.591224 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:50.619668 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:50.628466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 367e9138-2d81-4a46-8056-5d07308e918d up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:50.652488 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-4952c559-7e67-48f0-8209-d6e0840b8ad7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:50.654077 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-4952c559-7e67-48f0-8209-d6e0840b8ad7 service neutron] 10.222.0.95 "GET /v2.0/networks?id=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 HTTP/1.1" status: 200 len: 912 time: 0.5381858 May 24 10:00:50.678531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Allocated IP 172.27.0.64 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/f981d8ee-64c4-43a8-ac03-61f54f5cc98d) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:50.820695 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 367e9138-2d81-4a46-8056-5d07308e918d triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:50.837716 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:50.946966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.067070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ed75d389-b3db-45f5-bf82-87fe5209a3e9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:51.097897 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-efe9f992-d3d5-4a9e-a8f7-d03c6d79526e req-453b5f23-9f83-484f-9930-8576ec55074b tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:51.099679 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-efe9f992-d3d5-4a9e-a8f7-d03c6d79526e req-453b5f23-9f83-484f-9930-8576ec55074b tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.1471720 May 24 10:00:51.165726 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-efe9f992-d3d5-4a9e-a8f7-d03c6d79526e req-023a72cd-76fc-4d4d-988d-2017cc0a8550 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0458813 May 24 10:00:51.206621 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.219131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-a3fab0b9-d729-4909-a56d-b6bd8b57023b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:51.219131 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-a3fab0b9-d729-4909-a56d-b6bd8b57023b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-securitygroup--1492441025&fields=id&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 267 time: 1.1054850 May 24 10:00:51.252056 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:51.282248 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.308502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:51.345498 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:51.369350 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:51.369927 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.375939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.395681 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:51.399185 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:51.403660 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:51.411698 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=f981d8ee-64c4-43a8-ac03-61f54f5cc98d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:51.412152 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d180e909-d2e2-44ed-9742-4c0a7e9f84af,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:51.419877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:51.425428 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:51.433188 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=2ee76931177b477f8c6eafa067b78c15&name=default HTTP/1.1" status: 200 len: 2979 time: 1.9410863 May 24 10:00:51.469202 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51582) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:51.576145 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.608970 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-49a6c0f3-bfb9-4366-8d88-7323e5eec391 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:51.691453 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.708786 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=efa6f3438def4305baf64679ad31ac78&name=default HTTP/1.1" status: 200 len: 2979 time: 1.8313417 May 24 10:00:51.710912 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51592) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:51.713198 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=246f6ca7-bf19-4cb4-a0b4-8c1cd04ffa67,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:51.718194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:51.735490 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b3c342d3-9137-4dfa-b0d8-ba7bc738584f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:51.739169 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b3c342d3-9137-4dfa-b0d8-ba7bc738584f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 817 time: 0.4961486 May 24 10:00:51.769353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.771358 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-82f1df39-ddeb-49d0-bce7-867affec85dc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/quotas/c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 375 time: 0.0233760 May 24 10:00:51.818768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.883193 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.905790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.946450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:51.958440 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port ed75d389-b3db-45f5-bf82-87fe5209a3e9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ed75d389-b3db-45f5-bf82-87fe5209a3e9,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:51.987338 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.051943 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.061163 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:52.069533 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-abf81f0a-f850-4cfa-951d-6c457cb61acf tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2931139 May 24 10:00:52.232064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.250827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:52.264997 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d180e909-d2e2-44ed-9742-4c0a7e9f84af,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:52.271487 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:52.296818 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.307532 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:52.318424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.260s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:52.318542 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d9590a8e-2c43-485b-bcd9-e39a0b12bbf1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:52.339440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51614) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:52.401613 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.414598 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=246f6ca7-bf19-4cb4-a0b4-8c1cd04ffa67,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:52.467362 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:52.475103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Usage tracker for resource:security_group_rule and tenant:b02e0f18ca464e11afed761328595853 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:52.481229 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:52.510267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:52.537427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.221s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:52.567738 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-d947de5b-785e-479b-bec3-2a19770db30b service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c HTTP/1.1" status: 200 len: 217 time: 0.2347748 May 24 10:00:52.600467 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.db.resource_extend [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] It took 0.21 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 10:00:52.626062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.631289 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/246f6ca7-bf19-4cb4-a0b4-8c1cd04ffa67 HTTP/1.1" status: 204 len: 168 time: 0.9150510 May 24 10:00:52.633546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.638564 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/d180e909-d2e2-44ed-9742-4c0a7e9f84af HTTP/1.1" status: 204 len: 168 time: 1.1672804 May 24 10:00:52.649207 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:52.693335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f766571d-c79c-483d-89a0-68e180b44be5 req-e2140e4e-3050-460e-ad46-355fc8bad74e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:52.703134 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f766571d-c79c-483d-89a0-68e180b44be5 req-e2140e4e-3050-460e-ad46-355fc8bad74e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.2947018 May 24 10:00:52.778121 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f766571d-c79c-483d-89a0-68e180b44be5 req-88ca2da0-e075-4b68-a1dd-13307da60f05 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0732222 May 24 10:00:52.813862 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.148484 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.148752 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.155052 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:53.158478 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:53.221201 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b02e0f18ca464e11afed761328595853&name=default HTTP/1.1" status: 200 len: 2979 time: 0.8785844 May 24 10:00:53.224486 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.224647 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51626) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:53.251821 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=f981d8ee-64c4-43a8-ac03-61f54f5cc98d,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:53.255351 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:53.266780 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0a24e3d4-aeb2-4dbc-a44a-0ee9e2969df0,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:53.272432 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:53.308357 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.335675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.418117 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.419890 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 4.3770211 May 24 10:00:53.502712 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-e61d84a1-7684-444b-8a72-24686cb8d747 service neutron] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:00:53.503766 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-e61d84a1-7684-444b-8a72-24686cb8d747 service neutron] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8584237 May 24 10:00:53.504951 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.510328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:53.543597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0a24e3d4-aeb2-4dbc-a44a-0ee9e2969df0,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:53.551564 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:53.589155 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:53.593939 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/0a24e3d4-aeb2-4dbc-a44a-0ee9e2969df0 HTTP/1.1" status: 204 len: 168 time: 0.3629279 May 24 10:00:53.718715 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-942755d6-8dea-44f3-ae85-6855b8540956 service neutron] 10.222.0.67 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1982331 May 24 10:00:53.954726 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-3a373360-9322-4fd4-8b96-704e7f3153af tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:53.989845 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b0fe7ff7-7224-49ca-aaa4-b859e8d8f00c service neutron] 10.222.0.67 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2647023 May 24 10:00:53.991953 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51656) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:54.015341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.036046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:54.040970 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Usage tracker for resource:security_group_rule and tenant:cc7a148c2d3248339e88349661863b81 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:54.044226 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:54.051079 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-3a373360-9322-4fd4-8b96-704e7f3153af tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:54.052382 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-3a373360-9322-4fd4-8b96-704e7f3153af tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-securitygroup--1286635211&fields=id&tenant_id=3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 267 time: 0.1442263 May 24 10:00:54.053370 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:54.282046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-9286705f-7f33-46d4-a7a3-934e4804bdba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:54.282775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-9286705f-7f33-46d4-a7a3-934e4804bdba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:54.303799 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ed75d389-b3db-45f5-bf82-87fe5209a3e9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:00:54.341232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.406219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-9286705f-7f33-46d4-a7a3-934e4804bdba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:54.411494 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-9286705f-7f33-46d4-a7a3-934e4804bdba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 810 time: 0.3479297 May 24 10:00:54.435747 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-d2c76433-f61d-4df8-9b57-47463f20e5c0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/quotas/3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 375 time: 0.0179663 May 24 10:00:54.487300 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.501956 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port ed75d389-b3db-45f5-bf82-87fe5209a3e9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:54.502343 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.503719 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-02849df7-fda3-4580-8b17-1a4449d94978 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:54.562848 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=cc7a148c2d3248339e88349661863b81&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5698373 May 24 10:00:54.565770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51688) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:54.567499 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4fc65e6c-308d-4b17-9c25-a35658370ee3,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:54.577092 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:54.634239 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f05f661d-d03d-43fb-be27-c265a4e64296 req-a249dda7-c5c9-47a1-a0e5-65c455ee87dd tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:54.635855 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f05f661d-d03d-43fb-be27-c265a4e64296 req-a249dda7-c5c9-47a1-a0e5-65c455ee87dd tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.1795766 May 24 10:00:54.649905 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-6280c57e-3613-4025-944e-d4028e1e21fe tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2101028 May 24 10:00:54.761000 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.773807 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51702) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:54.798344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:54.867007 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-6a79bc53-c73c-416d-8d5d-9554ca9bbf8e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:54.867192 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f05f661d-d03d-43fb-be27-c265a4e64296 req-f96e5960-2654-4512-8b68-eb060c7fc1fe tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2198675 May 24 10:00:54.869098 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-6a79bc53-c73c-416d-8d5d-9554ca9bbf8e service neutron] 10.222.0.95 "GET /v2.0/subnets?id=b28ebd86-e35c-4b85-b6ba-3336770f64e8 HTTP/1.1" status: 200 len: 852 time: 0.1677933 May 24 10:00:54.941685 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-ebfe7c90-0113-4563-aa5c-35b4d4138de7 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0514407 May 24 10:00:54.951872 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-fc19258f-c13f-4d19-9b11-921eca1078dc service neutron] 10.222.0.67 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2884893 May 24 10:00:54.982797 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4a8fcc4f-3cd4-4558-8416-afbfa9dac578 req-67f4d57f-a36e-4013-b33b-8a6e8ca9bce3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.0582504 May 24 10:00:54.994764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:55.005579 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:55.018745 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4fc65e6c-308d-4b17-9c25-a35658370ee3,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:55.022230 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:55.024376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:55.031036 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/4fc65e6c-308d-4b17-9c25-a35658370ee3 HTTP/1.1" status: 204 len: 168 time: 0.4575737 May 24 10:00:55.124736 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:55.291322 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:55.309326 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-944f0a4b-8a59-44db-9971-686d94f285fc service neutron] 10.222.0.67 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3563833 May 24 10:00:55.311781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Usage tracker for resource:security_group_rule and tenant:386fc9a71b9a4a078dea28af6d42eb62 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:00:55.317571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:00:55.334097 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:55.397132 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-dbae47d9-2d31-4c38-9b9d-b8b6732066a4 service neutron] 10.222.0.95 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3392591 May 24 10:00:55.591776 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-1857bb42-1a21-4a0d-aadb-d32236757711 req-20b9e954-950c-4d8f-8c6b-08cfcd596823 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:55.599192 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb471aa4-e416-4efb-bb55-56683bd825ea req-8c77dc7a-f579-4036-9994-21062390300c service neutron] 10.222.0.95 "GET /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1956460 May 24 10:00:55.601725 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:55.949184 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:55.956916 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:55.976379 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-1857bb42-1a21-4a0d-aadb-d32236757711 req-20b9e954-950c-4d8f-8c6b-08cfcd596823 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:55.979658 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1857bb42-1a21-4a0d-aadb-d32236757711 req-20b9e954-950c-4d8f-8c6b-08cfcd596823 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.4608972 May 24 10:00:56.074407 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=386fc9a71b9a4a078dea28af6d42eb62&name=default HTTP/1.1" status: 200 len: 2979 time: 1.2949071 May 24 10:00:56.074407 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=70777ed1-078f-4b0e-a35c-f54e6f09e06a,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:56.076577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 51760) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:56.081407 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:56.111176 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f08bf2ea-f89a-4387-8520-81b4bc27033d req-de51cc5f-780d-4b3a-bb77-d82383f2d18a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.0479140 May 24 10:00:56.163335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:56.208603 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:56.242008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9bc2e8b-0a40-451e-948b-7c36dc0d96fd req-b2da3b27-0d72-47fd-9e2c-01836bf3b50c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:56.242008 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a9bc2e8b-0a40-451e-948b-7c36dc0d96fd req-b2da3b27-0d72-47fd-9e2c-01836bf3b50c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.2351265 May 24 10:00:56.310221 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a9bc2e8b-0a40-451e-948b-7c36dc0d96fd req-8920b923-1620-4612-b436-04c5b0cba03f tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0642264 May 24 10:00:56.520501 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:56.527213 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:56.544428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=70777ed1-078f-4b0e-a35c-f54e6f09e06a,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:56.547093 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:56.560561 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:56.567148 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/70777ed1-078f-4b0e-a35c-f54e6f09e06a HTTP/1.1" status: 204 len: 168 time: 0.4842324 May 24 10:00:57.141942 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6072c8b9-20c0-4aa9-9b71-2f2d7226a295 req-6b4eb7e9-ece0-4a0c-bf10-42cd544f4d99 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.0488851 May 24 10:00:57.300051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ed4ae609-d0c4-40bb-b3da-e39efcdeb700 req-f55eb1a1-fddb-43a2-9edd-db424e4624ec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:57.377297 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ed4ae609-d0c4-40bb-b3da-e39efcdeb700 req-f55eb1a1-fddb-43a2-9edd-db424e4624ec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:57.378634 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ed4ae609-d0c4-40bb-b3da-e39efcdeb700 req-f55eb1a1-fddb-43a2-9edd-db424e4624ec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.1317673 May 24 10:00:57.590155 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-770f8e07-968a-4f5f-ab6f-9c219025941c req-dadd7b60-6321-40b1-b795-db5a09b7871e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:57.590155 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-770f8e07-968a-4f5f-ab6f-9c219025941c req-dadd7b60-6321-40b1-b795-db5a09b7871e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.1388328 May 24 10:00:57.648573 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-770f8e07-968a-4f5f-ab6f-9c219025941c req-73460273-6ebf-41e9-b10a-eea4124bc9b6 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0525720 May 24 10:00:57.656676 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-5315df5d-8d1f-483b-a479-3eeffd54a327 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:57.657472 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-5315df5d-8d1f-483b-a479-3eeffd54a327 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:57.806786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-5315df5d-8d1f-483b-a479-3eeffd54a327 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:57.808163 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-5315df5d-8d1f-483b-a479-3eeffd54a327 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 817 time: 0.3577898 May 24 10:00:57.862466 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-c96f7927-1a27-4260-a229-c8a8f3e3db9d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:57.876601 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-c96f7927-1a27-4260-a229-c8a8f3e3db9d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:57.879879 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-c96f7927-1a27-4260-a229-c8a8f3e3db9d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 380 time: 0.0609117 May 24 10:00:57.889424 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'port': {'device_id': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'admin_state_up': True, 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'security_groups': ['d4764188-82ae-444d-ae05-a4e09e8d56dd']}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:58.071622 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:58.082972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:58.154989 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Made reservation on behalf of c42fc1bb2ae6477fa7b7c9261f2d50bb for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:58.162681 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:58.237997 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.95', 58142) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:00:58.335174 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5ca2bb84-dbc5-4495-b700-aeff3d50afbb req-57c0d733-475d-45dc-bd2e-c41e090a63ff tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.0625322 May 24 10:00:58.431223 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Allocated IP 172.20.0.12 (c1fb75f5-deee-4236-9b04-1b6b5e617606/327631ed-b242-499d-854c-7f681e3bd80e/6fec3696-3d1d-4762-9674-fe87fd37e211) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:58.461698 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-41b86fce-6eff-47a1-8ce8-30c8fd327a4f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:58.465076 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-41b86fce-6eff-47a1-8ce8-30c8fd327a4f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 810 time: 0.2259774 May 24 10:00:58.540913 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-d91b726e-aa20-4d05-986d-3fd738e16533 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:00:58.596940 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 35e9079c-497b-4448-a787-0fdee4f4fc4c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:00:58.611577 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a06bf092-773f-438a-85d8-20920f81a8cb req-d91b726e-aa20-4d05-986d-3fd738e16533 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:00:58.613682 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-d91b726e-aa20-4d05-986d-3fd738e16533 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 380 time: 0.1356244 May 24 10:00:58.628922 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0e479783-3047-4efb-82c9-e1135ca3df93 req-1c18053d-01e6-4996-b9d1-f78407c3d6ce tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.0654285 May 24 10:00:58.638903 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'port': {'device_id': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'admin_state_up': True, 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'security_groups': ['c864d029-e730-4251-9cfd-084868a9275d']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:58.742368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:58.767843 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Transition to ACTIVE for port object 6fec3696-3d1d-4762-9674-fe87fd37e211 will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:58.841071 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c,segment=NetworkSegment(deba89ad-e4b9-43a6-b2c3-6904c93466bf),segment_id=deba89ad-e4b9-43a6-b2c3-6904c93466bf)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:58.866660 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '35e9079c-497b-4448-a787-0fdee4f4fc4c', 'network_id': '1c6bd2e7-cf1c-4cda-aa53-7f057dec1138', 'port_id': '35e9079c-497b-4448-a787-0fdee4f4fc4c', 'mac_address': 'fa:16:3e:5c:ed:5e', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 55, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b28ebd86-e35c-4b85-b6ba-3336770f64e8', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:00:58.867598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:00:58.881887 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:00:58.902314 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a065572-d286-4299-9b2a-db2999f7a39d req-181d0fed-6157-4063-bc9f-56d683df0556 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:58.903824 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4a065572-d286-4299-9b2a-db2999f7a39d req-181d0fed-6157-4063-bc9f-56d683df0556 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 943 time: 0.1266971 May 24 10:00:58.924122 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:58.944473 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 35e9079c-497b-4448-a787-0fdee4f4fc4c {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:00:58.957457 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4a065572-d286-4299-9b2a-db2999f7a39d req-6764f89a-5951-4771-bfd5-220d3d16475c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436647 May 24 10:00:58.969744 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c,segment=NetworkSegment(deba89ad-e4b9-43a6-b2c3-6904c93466bf),segment_id=deba89ad-e4b9-43a6-b2c3-6904c93466bf)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:00:58.988909 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Made reservation on behalf of 3f91ad6930924a8d81ee21ec1c490828 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:00:58.990418 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:58.994307 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.005593 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:59.074849 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:59.078245 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:59.085360 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.088760 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:59.135966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:59.141951 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:59.161257 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:59.161936 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.179056 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:59.205762 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Allocated IP 172.20.0.4 (e28486c2-462f-406b-8c64-3246ae76f56c/59e96feb-f5d8-4b1e-b08d-b93c34f5461c/b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:00:59.255892 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:59.270726 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.182s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:00:59.309787 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.4243207 May 24 10:00:59.326833 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Request body: {'port': {'device_id': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:00:59.345025 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['35e9079c-497b-4448-a787-0fdee4f4fc4c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:00:59.556022 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.562614 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.569502 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7ebdaa2e-acd5-4c9a-99b6-98adef75eb91 None None] Provisioning for port 6fec3696-3d1d-4762-9674-fe87fd37e211 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:00:59.582423 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Transition to ACTIVE for port object b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:00:59.591881 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7ebdaa2e-acd5-4c9a-99b6-98adef75eb91 None None] Provisioning complete for port 6fec3696-3d1d-4762-9674-fe87fd37e211 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:00:59.596935 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ebdaa2e-acd5-4c9a-99b6-98adef75eb91 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.635264 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b0facf7c-5f95-42a2-b392-b0ff61c06d7c req-22950eeb-3cbb-4086-9d9a-95aec09fe6f1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.0442533 May 24 10:00:59.666381 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-7ebdaa2e-acd5-4c9a-99b6-98adef75eb91 None None] Port 6fec3696-3d1d-4762-9674-fe87fd37e211 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:00:59.806773 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.813163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:00:59.886678 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Network e28486c2-462f-406b-8c64-3246ae76f56c is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:00:59.890036 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:00:59.894299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:00:59.900201 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:00:59.900756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:00:59.910520 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:00:59.918857 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:00.005754 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3874607 May 24 10:01:00.018906 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Request body: {'port': {'device_id': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:00.109074 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f09376ba-28af-4a5e-ad7f-e8d0bf23911d req-4e89a3c9-65af-4621-89d3-c9c2108cbf3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:00.109826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-f09376ba-28af-4a5e-ad7f-e8d0bf23911d req-4e89a3c9-65af-4621-89d3-c9c2108cbf3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:00.224280 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-f09376ba-28af-4a5e-ad7f-e8d0bf23911d req-4e89a3c9-65af-4621-89d3-c9c2108cbf3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:00.230989 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f09376ba-28af-4a5e-ad7f-e8d0bf23911d req-4e89a3c9-65af-4621-89d3-c9c2108cbf3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 932 time: 0.4531026 May 24 10:01:00.263165 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-b5686216-dbcf-45c8-b5c1-dd8844f28043 None None] Provisioning for port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:00.270679 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-b5686216-dbcf-45c8-b5c1-dd8844f28043 None None] Provisioning complete for port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:00.272276 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b5686216-dbcf-45c8-b5c1-dd8844f28043 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.279055 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:00.297469 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f09376ba-28af-4a5e-ad7f-e8d0bf23911d req-90ea0ddb-f505-4a0f-9b7c-472b5e9e0a5e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0655079 May 24 10:01:00.305545 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.344220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Transition to ACTIVE for port object 6fec3696-3d1d-4762-9674-fe87fd37e211 will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:01:00.395612 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-b5686216-dbcf-45c8-b5c1-dd8844f28043 None None] Port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:01:00.420739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.426158 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:00.434539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3609795c-07ec-44ca-b6f7-1fbb8ac4911e req-c9a3f21c-b6e4-413f-8789-9b9f10a24822 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:00.436059 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3609795c-07ec-44ca-b6f7-1fbb8ac4911e req-c9a3f21c-b6e4-413f-8789-9b9f10a24822 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 944 time: 0.3300395 May 24 10:01:00.445947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.570364 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 35e9079c-497b-4448-a787-0fdee4f4fc4c up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:01:00.686675 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3609795c-07ec-44ca-b6f7-1fbb8ac4911e req-0dc48b1c-4ffa-4c7a-a969-6242139e532a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2423384 May 24 10:01:00.766377 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 35e9079c-497b-4448-a787-0fdee4f4fc4c completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:00.777317 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 35e9079c-497b-4448-a787-0fdee4f4fc4c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:00.777696 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.810693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Attempting to bind port 6fec3696-3d1d-4762-9674-fe87fd37e211 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:01:00.811226 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Attempting to bind port 6fec3696-3d1d-4762-9674-fe87fd37e211 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:01:00.811385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Attempting to bind port 6fec3696-3d1d-4762-9674-fe87fd37e211 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:00.818852 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:00.831020 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.405s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:00.840766 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Port 6fec3696-3d1d-4762-9674-fe87fd37e211 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:01:00.841972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Attempting to bind port 6fec3696-3d1d-4762-9674-fe87fd37e211 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:00.854640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:01:00.855143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Checking segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:00.855761 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Bound using segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:01:00.856202 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Bound port: 6fec3696-3d1d-4762-9674-fe87fd37e211, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:01:00.856779 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:00.979192 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.017088 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:01.038356 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:01:01.042492 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:01:01.085186 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69ff9044-5a30-4292-8b81-1f5e94eaf186 req-e37e0dee-55bf-41a1-8851-4e1b9bf2aae6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:01.089844 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c,segment=NetworkSegment(deba89ad-e4b9-43a6-b2c3-6904c93466bf),segment_id=deba89ad-e4b9-43a6-b2c3-6904c93466bf)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:01.092908 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-69ff9044-5a30-4292-8b81-1f5e94eaf186 req-e37e0dee-55bf-41a1-8851-4e1b9bf2aae6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 931 time: 0.2157931 May 24 10:01:01.100813 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:01.112728 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.134992 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Transition to ACTIVE for port object b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:01:01.140330 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.146137 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-69ff9044-5a30-4292-8b81-1f5e94eaf186 req-5ecc81ee-7b5d-4709-b451-867da7fc90e1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0521302 May 24 10:01:01.151044 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:01.160321 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.163869 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:01.200122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.207002 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:01.224857 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:01.224857 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.224857 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] 10.222.0.95 "PUT /v2.0/ports/6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 1094 time: 1.9003682 May 24 10:01:01.224857 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:01.250038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:01.253376 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:01.413178 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:01.443845 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.267s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:01.444288 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'status': 'completed', 'code': 200} May 24 10:01:01.544830 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-04523dcd-0868-4fc4-afd5-213d253fc684 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:01.550348 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-04523dcd-0868-4fc4-afd5-213d253fc684 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1097 time: 0.1924620 May 24 10:01:01.624627 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Pushing event updated for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:01.630277 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.423s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:01.636003 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Attempting to bind port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:01:01.636455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Attempting to bind port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:01:01.636999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Attempting to bind port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:01.646276 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f on network e28486c2-462f-406b-8c64-3246ae76f56c not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:01:01.646693 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Attempting to bind port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:01.654053 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 0, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:01:01.654532 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Checking segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:01.655907 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Bound using segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:01:01.656477 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Bound port: b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:01:01.656937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:01.723628 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c', 'code': 200} May 24 10:01:01.826299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:01:01.836706 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:01:01.850037 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-c042401c-e718-4842-b61a-6890a218ba38 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 217 time: 0.2937558 May 24 10:01:01.968070 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-2fe60f21-4785-4d09-88f3-735381b90fdb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:01.969617 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-2fe60f21-4785-4d09-88f3-735381b90fdb service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.1111467 May 24 10:01:01.985451 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:02.025513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:02.028892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e491b90-2a2a-4756-b791-86919bafe98d req-dfd62431-34b9-41a2-8352-eef80e237960 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.030470 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6e491b90-2a2a-4756-b791-86919bafe98d req-dfd62431-34b9-41a2-8352-eef80e237960 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.3103147 May 24 10:01:02.104947 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6e491b90-2a2a-4756-b791-86919bafe98d req-c17462e6-b39b-40c8-b67f-d8091cd81d21 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0607741 May 24 10:01:02.122294 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-d522ef76-c5e6-4cbf-9656-be08eb68e50b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.124595 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-d522ef76-c5e6-4cbf-9656-be08eb68e50b service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1468194 May 24 10:01:02.131590 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.95', 58147) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:02.190367 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.198227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:02.200931 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] 10.222.0.95 "PUT /v2.0/ports/b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1093 time: 2.1832950 May 24 10:01:02.201577 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:02.310569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Pushing event updated for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:02.314311 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.293s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:02.387020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 51984) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:02.429202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c4963ee-fc94-43a5-9490-97747b550ad3 req-a2f4121a-e3d1-4d5b-846f-13e88d35952c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.437599 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c4963ee-fc94-43a5-9490-97747b550ad3 req-a2f4121a-e3d1-4d5b-846f-13e88d35952c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.5529492 May 24 10:01:02.536296 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c4963ee-fc94-43a5-9490-97747b550ad3 req-92c2b5e9-a6c5-4159-82a5-220b0b4c1537 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0950263 May 24 10:01:02.552576 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-7fbf1c21-dc4d-4674-8e91-3b6781e38ed5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.554466 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-7fbf1c21-dc4d-4674-8e91-3b6781e38ed5 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1096 time: 0.2380795 May 24 10:01:02.564028 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-c6388e0d-94d8-4f00-861d-02f991734798 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4271262 May 24 10:01:02.618697 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-ecfadbba-2446-4009-9dd2-5b68ea7abd76 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.620183 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-ecfadbba-2446-4009-9dd2-5b68ea7abd76 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 1098 time: 0.4425189 May 24 10:01:02.633335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52aa4465-f882-4084-b184-5a42ebb7daa3 req-d34af5b9-12e4-47ed-af38-8f3d2313a624 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.633335 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-52aa4465-f882-4084-b184-5a42ebb7daa3 req-d34af5b9-12e4-47ed-af38-8f3d2313a624 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 943 time: 0.2897007 May 24 10:01:02.672500 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-50651129-d271-4db9-80ec-111ee4782224 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 217 time: 0.1098959 May 24 10:01:02.715185 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:02.751038 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-bcb4cdf5-6ca8-4fc2-bf5e-2e830cf74f19 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.752721 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-bcb4cdf5-6ca8-4fc2-bf5e-2e830cf74f19 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.0723894 May 24 10:01:02.808502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:02.834010 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:02.845286 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:02.845871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Usage tracker for resource:security_group_rule and tenant:ea4946fb0c044fb8ac328ee9fc158b53 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:02.852649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:02.853892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Usage tracker for resource:security_group_rule and tenant:b360aed94abc461d99b3f1b169616d0d is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:02.857284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:02.866525 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:02.882497 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:02.896975 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-52aa4465-f882-4084-b184-5a42ebb7daa3 req-88a8cfaa-8568-44ba-a007-4f6b0bad22dc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1711817 May 24 10:01:02.929712 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06bf092-773f-438a-85d8-20920f81a8cb req-62a22c2e-8ace-4a67-9074-bfe4e8ea0a1f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:02.931773 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-62a22c2e-8ace-4a67-9074-bfe4e8ea0a1f service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1695225 May 24 10:01:02.940750 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ead8085e-8a12-4dcf-a884-0bbccb845a4f req-95b8884a-9c36-4707-8024-1ae68d62cb4f service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3696387 May 24 10:01:03.047767 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'status': 'completed', 'code': 200} May 24 10:01:03.129882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.151044 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.176213 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-54a906a4-a2a4-40f9-a069-3528937a2f71 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2325799 May 24 10:01:03.179329 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.188351 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-204d8f09-9897-4276-ad59-a968b84d8d20 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.196014 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-204d8f09-9897-4276-ad59-a968b84d8d20 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1097 time: 0.1122007 May 24 10:01:03.198399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.233190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:03.280102 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7b1b0482-4492-4192-99e0-3b73a4a174da None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.304143 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9a14a8da-b3df-45de-9124-6bfc1faf92de req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.6699257 May 24 10:01:03.373878 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a2d33946-07f2-490c-90d1-f4b5d2b7675c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:03.381952 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Pushing event updated for resources: {'SecurityGroup': ['ID=29fb4a8f-86bb-4114-a669-0deafee71ec8,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:03.389423 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b360aed94abc461d99b3f1b169616d0d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9987481 May 24 10:01:03.400964 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52032) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:03.426051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.194s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:03.469080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f7184bd9-91dd-4c1e-b22e-85a1e30e70e1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:03.473673 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:03.484804 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-c5ace767-16ba-4775-8891-efcae6a9ebaa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.486290 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-c5ace767-16ba-4775-8891-efcae6a9ebaa service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.2739546 May 24 10:01:03.514020 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a06bf092-773f-438a-85d8-20920f81a8cb req-55c95213-36eb-40b6-80b2-8adda14770a4 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2657607 May 24 10:01:03.562618 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-a3c70897-3e47-4ec8-bf18-12e4408e86ff service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 217 time: 0.0691848 May 24 10:01:03.571191 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6eb57898-1207-471d-8bfb-e5a2db0b4b73 req-7ff6e9f1-9658-43d4-8ce6-369d271a0ba2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.575097 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6eb57898-1207-471d-8bfb-e5a2db0b4b73 req-7ff6e9f1-9658-43d4-8ce6-369d271a0ba2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.2411458 May 24 10:01:03.585853 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.627232 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:03.677320 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6eb57898-1207-471d-8bfb-e5a2db0b4b73 req-7de238d2-e004-4ccf-9810-ebbad22c2d07 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0688221 May 24 10:01:03.683119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-75aad625-9242-4f98-a156-0e09783cc263 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.684483 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-75aad625-9242-4f98-a156-0e09783cc263 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.1082654 May 24 10:01:03.777544 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-a5b0d8fd-1a5a-47a3-9927-d4b3695d7ed9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.779167 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-a5b0d8fd-1a5a-47a3-9927-d4b3695d7ed9 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0866070 May 24 10:01:03.924302 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-fddb988e-dad7-4811-925f-d0acf2b4a14e service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1369414 May 24 10:01:03.927195 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-71723855-e264-4813-b800-1e7782d83f49 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.928568 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-71723855-e264-4813-b800-1e7782d83f49 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1096 time: 0.1885045 May 24 10:01:03.939524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46797f6d-f572-4bfe-8442-1de21422d72e req-9d076836-e775-4645-abe7-e4469de91796 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:03.940979 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-46797f6d-f572-4bfe-8442-1de21422d72e req-9d076836-e775-4645-abe7-e4469de91796 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.1510096 May 24 10:01:04.017460 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:04.024284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:04.143210 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f7184bd9-91dd-4c1e-b22e-85a1e30e70e1,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:04.152139 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-46797f6d-f572-4bfe-8442-1de21422d72e req-fc7c6779-1e81-40ff-9cdf-451441262d91 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1917729 May 24 10:01:04.152882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:04.165760 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:04.167032 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/f7184bd9-91dd-4c1e-b22e-85a1e30e70e1 HTTP/1.1" status: 204 len: 168 time: 0.7621408 May 24 10:01:04.241887 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-90846680-3ee5-4903-835d-427843405c3b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:04.243074 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab07c044-44ed-493d-abf8-27d5ebed0e18 req-6e7dcb4a-5e2a-41b0-97d1-0b179a611758 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3063021 May 24 10:01:04.243340 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-90846680-3ee5-4903-835d-427843405c3b service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.3014154 May 24 10:01:04.327087 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ea7f35-9e36-4eb1-b622-d76392745ebb req-be6779e3-9f9d-4d82-a14f-8be473be4f3c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:04.330561 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-93ea7f35-9e36-4eb1-b622-d76392745ebb req-be6779e3-9f9d-4d82-a14f-8be473be4f3c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 943 time: 0.2702017 May 24 10:01:04.380683 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-93ea7f35-9e36-4eb1-b622-d76392745ebb req-17141a48-184d-41fd-906a-a67b8a6ff58e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394137 May 24 10:01:04.878060 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b77e5aac-3c0f-4984-a30d-c81580424060 req-57b667cb-9ba2-45a6-9bf7-7da03f517768 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:04.879575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b77e5aac-3c0f-4984-a30d-c81580424060 req-57b667cb-9ba2-45a6-9bf7-7da03f517768 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0850160 May 24 10:01:04.928500 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52082) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:04.928910 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b77e5aac-3c0f-4984-a30d-c81580424060 req-6a3943b8-b71b-44c9-98cc-2577029fec62 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432794 May 24 10:01:04.952109 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:04.969720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:04.974989 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Usage tracker for resource:security_group_rule and tenant:9072eb8b04b342fca38b584623103a5f is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:04.978365 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:04.985700 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:05.069803 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.205114 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.214497 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0e7235ba-33c7-422a-8374-cad194b529ca admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:05.253263 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=9072eb8b04b342fca38b584623103a5f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3189645 May 24 10:01:05.254908 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7d0e3aec-8ee6-416e-bf3e-55b5dd4b8266,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:05.256056 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52106) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:05.257871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:05.337389 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.376664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.385512 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c77cca81-e78e-4a1e-a0a2-af6607fa7187 req-7c1b87d9-774a-4d00-a290-89e8a3bbc5d9 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:05.393017 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c77cca81-e78e-4a1e-a0a2-af6607fa7187 req-7c1b87d9-774a-4d00-a290-89e8a3bbc5d9 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.1098068 May 24 10:01:05.450336 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c77cca81-e78e-4a1e-a0a2-af6607fa7187 req-5a151e9e-04cc-4723-8aa2-f253f870a25c tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0517170 May 24 10:01:05.542560 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.555274 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:05.574722 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7d0e3aec-8ee6-416e-bf3e-55b5dd4b8266,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:05.580741 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:05.589656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.589656 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/7d0e3aec-8ee6-416e-bf3e-55b5dd4b8266 HTTP/1.1" status: 204 len: 168 time: 0.3270235 May 24 10:01:05.621457 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9243623a-366a-4a79-874e-29ee0444be94 req-4d7bb9c2-dbe9-4f3a-8dad-93f6102d7d68 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:05.622183 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-9243623a-366a-4a79-874e-29ee0444be94 req-4d7bb9c2-dbe9-4f3a-8dad-93f6102d7d68 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:05.702914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-9243623a-366a-4a79-874e-29ee0444be94 req-4d7bb9c2-dbe9-4f3a-8dad-93f6102d7d68 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:05.709621 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9243623a-366a-4a79-874e-29ee0444be94 req-4d7bb9c2-dbe9-4f3a-8dad-93f6102d7d68 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 943 time: 0.2018945 May 24 10:01:05.742699 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:05.752857 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9243623a-366a-4a79-874e-29ee0444be94 req-39413bc8-d402-4771-9099-9f22718c8523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0422587 May 24 10:01:05.765941 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6544ceb8-d77a-4c06-8702-602c6f25e58d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:06.136655 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22636219-8af6-48c8-93e5-187120a5651c req-8d9f51aa-0cbb-46cc-b1e4-45beefa28bb4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:06.141846 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-22636219-8af6-48c8-93e5-187120a5651c req-8d9f51aa-0cbb-46cc-b1e4-45beefa28bb4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0862336 May 24 10:01:06.188919 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-22636219-8af6-48c8-93e5-187120a5651c req-f8882670-6504-4735-b5a2-88da6ef2e992 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0444953 May 24 10:01:06.605975 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52182) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:06.646391 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c5a9127-495b-46f6-86bf-ead626dd6045 req-b9d57ab8-8e7c-40f8-b272-67c79343cf1f tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:06.648501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c5a9127-495b-46f6-86bf-ead626dd6045 req-b9d57ab8-8e7c-40f8-b272-67c79343cf1f tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0829830 May 24 10:01:06.701650 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c5a9127-495b-46f6-86bf-ead626dd6045 req-f886f89a-19d9-4152-991c-9b6dc19bf15a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0460112 May 24 10:01:06.847858 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:06.898312 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:06.911752 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Usage tracker for resource:security_group_rule and tenant:5e36f919707345ae8f8a82c0e2bafe5d is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:06.916947 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:06.924418 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:06.981378 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fe82a5f-f031-4f97-9940-538048f8f2bf req-b2fb491e-e500-44d0-8cde-11ef31f1ba5f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:06.982858 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2fe82a5f-f031-4f97-9940-538048f8f2bf req-b2fb491e-e500-44d0-8cde-11ef31f1ba5f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 943 time: 0.1207256 May 24 10:01:07.011777 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:01:07.062319 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2fe82a5f-f031-4f97-9940-538048f8f2bf req-2aab64a2-e4cc-4750-931d-1f223ea66baf tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0727754 May 24 10:01:07.075786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.110605 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-bb20d5ea-3845-47d6-8631-6c34b50f0ffe service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 217 time: 0.1097271 May 24 10:01:07.172373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.194039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-68e51f78-1c65-477c-ab3c-b2fe6784bf36 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:07.195699 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-68e51f78-1c65-477c-ab3c-b2fe6784bf36 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.0756993 May 24 10:01:07.198228 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-715af245-7268-42af-aaf2-60c1056b11fc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:07.200092 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:07.214969 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:01:07.258906 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.288370 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e5725aab-9ecf-4468-ab1f-7782dd498877,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:07.290112 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.301136 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:07.319234 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-7a509229-8a51-409e-8d0e-bf2be79b25e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:07.320182 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:07.321182 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-7a509229-8a51-409e-8d0e-bf2be79b25e7 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1154022 May 24 10:01:07.324505 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=5e36f919707345ae8f8a82c0e2bafe5d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7173247 May 24 10:01:07.324764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52206) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:07.340133 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:07.423057 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.424157 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:01:07.427751 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:07.530135 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:07.543833 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:07.561455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.593451 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-3c70cf19-49b3-455f-8fe2-a4c9e5315c84 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2614021 May 24 10:01:07.667108 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.695108 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:07.710793 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c8f85e1-f22b-4576-980f-d467833a2c35 req-de2b63f5-9ed7-4211-93df-6bf6d47a1ca7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:07.714418 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'port_id': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'mac_address': 'fa:16:3e:0b:f6:e6', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 61, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:01:07.718348 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c8f85e1-f22b-4576-980f-d467833a2c35 req-de2b63f5-9ed7-4211-93df-6bf6d47a1ca7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.3557613 May 24 10:01:07.806524 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:07.888404 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c8f85e1-f22b-4576-980f-d467833a2c35 req-ec92eded-4b4e-4dc7-8f37-171e3d78afd4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1689343 May 24 10:01:07.901093 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:07.906179 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6165aac6-5132-417c-aedd-1c95c686bff1 req-eabcaa83-875b-4136-be85-45ab341427e2 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2896216 May 24 10:01:07.920727 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:08.005023 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-156efa86-1cc1-422a-8dde-e19ac630a3b8 req-dabb7be6-2aa2-4109-bbdc-5191da4a229d tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:08.005971 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:08.006156 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:08.009932 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-156efa86-1cc1-422a-8dde-e19ac630a3b8 req-dabb7be6-2aa2-4109-bbdc-5191da4a229d tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0890021 May 24 10:01:08.010054 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:08.017427 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3acd0c16-805d-467a-8981-92b0ea7a2ced admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:08.030834 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e5725aab-9ecf-4468-ab1f-7782dd498877,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:08.033340 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:08.038854 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:08.042549 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e5725aab-9ecf-4468-ab1f-7782dd498877 HTTP/1.1" status: 204 len: 168 time: 0.7135568 May 24 10:01:08.105227 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-156efa86-1cc1-422a-8dde-e19ac630a3b8 req-9d115448-3e53-4a28-9fd8-0193b6fc2fc8 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0890551 May 24 10:01:08.106080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:08.119414 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:08.222401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:01:08.270508 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['6fec3696-3d1d-4762-9674-fe87fd37e211'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:01:08.313719 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d2245a8-dc12-4eac-b2a6-3308d7f02fbf req-28f361b1-0934-4759-9057-f8fa1c26fb9e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:08.315509 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5d2245a8-dc12-4eac-b2a6-3308d7f02fbf req-28f361b1-0934-4759-9057-f8fa1c26fb9e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 944 time: 0.1162245 May 24 10:01:08.376008 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5d2245a8-dc12-4eac-b2a6-3308d7f02fbf req-186640e4-a212-4e17-b477-35d225e8b130 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0536096 May 24 10:01:08.737837 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:08.906510 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52266) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:08.926560 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:08.941002 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:08.945146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Usage tracker for resource:security_group_rule and tenant:ab37c0bf90b44826ba6dcb7f2b89d5a8 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:08.948690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:08.955488 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:09.043688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.109443 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.139664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:09.176541 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ab37c0bf90b44826ba6dcb7f2b89d5a8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2685258 May 24 10:01:09.183913 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52278) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:09.185522 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e64ac18e-b6a5-45bd-b9df-90528dd00823 req-f80524d2-de55-4eb0-b452-cb12083ac7b4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:09.195155 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e64ac18e-b6a5-45bd-b9df-90528dd00823 req-f80524d2-de55-4eb0-b452-cb12083ac7b4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 945 time: 0.1869950 May 24 10:01:09.203469 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=68e5e4e2-be7d-494a-80c0-e1a78b8bb686,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:09.208714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:09.263481 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e64ac18e-b6a5-45bd-b9df-90528dd00823 req-7eb4e439-dbb5-497f-8b9a-4642fc0c5c78 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0649424 May 24 10:01:09.395848 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.435848 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.464940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef0496b3-5059-42b7-886e-540eb5dff4ed req-9a0ba934-2a9f-462b-9597-bf49ec2b7507 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:09.464940 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ef0496b3-5059-42b7-886e-540eb5dff4ed req-9a0ba934-2a9f-462b-9597-bf49ec2b7507 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.1470072 May 24 10:01:09.527326 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ef0496b3-5059-42b7-886e-540eb5dff4ed req-089cf2ed-6792-47c1-b2cc-8668fa3b6c57 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0523927 May 24 10:01:09.609479 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a899d91-b4ed-481c-81d2-220691b4b4a7 req-4f94b53c-b174-44fa-86eb-2d78bf88b1c7 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:09.611128 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8a899d91-b4ed-481c-81d2-220691b4b4a7 req-4f94b53c-b174-44fa-86eb-2d78bf88b1c7 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 944 time: 0.1023498 May 24 10:01:09.618275 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.625475 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e1eff386-108d-4f53-a4b4-f869e4bf91b8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:09.642556 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=68e5e4e2-be7d-494a-80c0-e1a78b8bb686,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:09.654898 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:09.658925 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:09.660169 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/68e5e4e2-be7d-494a-80c0-e1a78b8bb686 HTTP/1.1" status: 204 len: 168 time: 0.4754667 May 24 10:01:09.669444 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8a899d91-b4ed-481c-81d2-220691b4b4a7 req-3586f6a8-eaa0-43a9-ab59-602cdce94b46 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0515778 May 24 10:01:10.466356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a47b2ebe-6e6d-4be3-9dd4-9ba5ec51d8cb req-64b0c68d-2842-4fe2-a6bd-62689819d89a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:10.467796 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a47b2ebe-6e6d-4be3-9dd4-9ba5ec51d8cb req-64b0c68d-2842-4fe2-a6bd-62689819d89a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 945 time: 0.0847313 May 24 10:01:10.526508 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a47b2ebe-6e6d-4be3-9dd4-9ba5ec51d8cb req-347db474-cb53-4ae5-8a71-567eeb9aec78 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0522881 May 24 10:01:10.640283 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:01:10.785200 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 6fec3696-3d1d-4762-9674-fe87fd37e211 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:10.791212 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 6fec3696-3d1d-4762-9674-fe87fd37e211 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:10.791777 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:10.895962 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6bf6067-b775-4c39-b29b-ccc3404f124f req-7a07ed53-7d79-42e4-9e5b-f9a281158121 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:10.897887 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c6bf6067-b775-4c39-b29b-ccc3404f124f req-7a07ed53-7d79-42e4-9e5b-f9a281158121 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 944 time: 0.1038558 May 24 10:01:10.956579 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:10.971689 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c6bf6067-b775-4c39-b29b-ccc3404f124f req-a8e1d19a-39be-4076-ae92-35866ba28bb3 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0657530 May 24 10:01:11.007116 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:11.020654 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:11.067901 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'code': 200} May 24 10:01:11.074374 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:11.086775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:01:11.129908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:11.230287 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:11.235144 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:11.342154 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:11.347639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:11.348026 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:11.483969 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:11.520824 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:11.645894 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:11.648940 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:11.730911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:11.741923 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:11.747071 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9b60e45-48e7-4521-9c07-00b3868fcd46 req-0973e058-a110-4c80-b7c6-4bcbc34f1234 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:11.748878 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9b60e45-48e7-4521-9c07-00b3868fcd46 req-0973e058-a110-4c80-b7c6-4bcbc34f1234 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0915096 May 24 10:01:11.804908 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9b60e45-48e7-4521-9c07-00b3868fcd46 req-bd54e7b2-9d1c-48eb-a662-075ee3d1f2a6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0495124 May 24 10:01:12.176718 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b9693ea-f920-4b94-8d5e-8672bc5203bd req-e90097e6-ba3e-4666-9565-cdb37eadb11e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:12.180435 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3b9693ea-f920-4b94-8d5e-8672bc5203bd req-e90097e6-ba3e-4666-9565-cdb37eadb11e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0910087 May 24 10:01:12.241082 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3b9693ea-f920-4b94-8d5e-8672bc5203bd req-366f24cb-332b-49ea-8aac-8730608c5998 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0538805 May 24 10:01:12.880073 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52414) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:13.000921 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb7cfb27-4208-42fe-a6fc-c901ef9a0d43 req-a5df5f1c-554d-4665-93ca-fc519606d24d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:13.002760 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb7cfb27-4208-42fe-a6fc-c901ef9a0d43 req-a5df5f1c-554d-4665-93ca-fc519606d24d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0876200 May 24 10:01:13.015198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.029846 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:13.033675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Usage tracker for resource:security_group_rule and tenant:29d1163be65b47039ec8dfc7397014af is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:13.036456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:13.042678 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:13.048816 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb7cfb27-4208-42fe-a6fc-c901ef9a0d43 req-ed7072c7-f997-44a9-8768-8e76e0bbbb0e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0398617 May 24 10:01:13.069154 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:13.116967 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'code': 200} May 24 10:01:13.131462 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.210773 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.221635 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-efe2bc9f-5b0e-4292-83f9-6bae1c48c324 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:13.261153 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=29d1163be65b47039ec8dfc7397014af&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3798103 May 24 10:01:13.262103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e31b348b-f41c-4de3-aebd-f67ba79239c7,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:13.264361 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52426) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:13.265631 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:13.324723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.353518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.462945 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b7987c6-5592-4b1e-9d12-9f3650626552 req-b4cb7c8e-4c23-4da8-9a70-d7733ab84745 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:13.464404 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0b7987c6-5592-4b1e-9d12-9f3650626552 req-b4cb7c8e-4c23-4da8-9a70-d7733ab84745 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1135964 May 24 10:01:13.508995 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0b7987c6-5592-4b1e-9d12-9f3650626552 req-1cea2bc7-008f-43b0-af82-ec492a16eb23 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0382752 May 24 10:01:13.536687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.540412 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b240d4ce-cb3c-4338-84f7-568bcff314cc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:13.554635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e31b348b-f41c-4de3-aebd-f67ba79239c7,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:13.559026 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:13.570995 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:13.573076 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e31b348b-f41c-4de3-aebd-f67ba79239c7 HTTP/1.1" status: 204 len: 168 time: 0.3073452 May 24 10:01:14.126178 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52444) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:14.152273 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.169273 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:14.207611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Usage tracker for resource:security_group_rule and tenant:7c26627defde41979757a606175fdc03 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:14.215011 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:14.230513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:14.240166 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7cbb65ae-be5e-41d0-b475-1f53a96620e9 req-0477488d-d666-4a91-92bc-5b1efdadd02d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:14.241812 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7cbb65ae-be5e-41d0-b475-1f53a96620e9 req-0477488d-d666-4a91-92bc-5b1efdadd02d service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1099 time: 0.0907993 May 24 10:01:14.294146 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7735e34e-e0b2-4319-9e7b-192140359f45 req-34a27b61-d326-4e7a-9af7-5a20db00f4b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:14.295674 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7735e34e-e0b2-4319-9e7b-192140359f45 req-34a27b61-d326-4e7a-9af7-5a20db00f4b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1239593 May 24 10:01:14.299379 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7cbb65ae-be5e-41d0-b475-1f53a96620e9 req-ccc0e2b5-805a-4b06-b409-d691b6110653 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0493567 May 24 10:01:14.374953 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7735e34e-e0b2-4319-9e7b-192140359f45 req-3fc426a6-7b9d-4663-8ce0-9ddb8c61f803 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0722694 May 24 10:01:14.382628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.463183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.473665 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:14.520521 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=7c26627defde41979757a606175fdc03&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3932388 May 24 10:01:14.521787 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=be76d0e0-c8fa-4d91-8d33-0e1d5caaa9eb,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:14.524170 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52456) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:14.527336 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:14.585117 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b5cfd3d-63bf-4468-845c-0fa5aee4868b req-f064059f-dc24-4a20-a5cc-180df9376122 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:14.586590 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3b5cfd3d-63bf-4468-845c-0fa5aee4868b req-f064059f-dc24-4a20-a5cc-180df9376122 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.1052229 May 24 10:01:14.590693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.619304 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.635710 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3b5cfd3d-63bf-4468-845c-0fa5aee4868b req-2a3c345d-236a-4300-bdb6-436dee3896d6 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0415802 May 24 10:01:14.734509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df0f09a9-72f9-4620-a536-4b85f5c210ff req-502ef4de-29d5-4a77-89fe-9fcf670ec9a2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:14.735963 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-df0f09a9-72f9-4620-a536-4b85f5c210ff req-502ef4de-29d5-4a77-89fe-9fcf670ec9a2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0981815 May 24 10:01:14.783633 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-df0f09a9-72f9-4620-a536-4b85f5c210ff req-7b23f111-2759-42bd-add0-c2f3859332f5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0419989 May 24 10:01:14.846330 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.851689 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:14.862056 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=be76d0e0-c8fa-4d91-8d33-0e1d5caaa9eb,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:14.867095 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:14.868917 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:14.870468 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/be76d0e0-c8fa-4d91-8d33-0e1d5caaa9eb HTTP/1.1" status: 204 len: 168 time: 0.3442938 May 24 10:01:15.600785 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8aba3bdd-a8ed-4dc3-8571-5ea3dff0a2e0 req-e8101c9d-a67f-4ffe-8031-a35667532b29 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:15.602487 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8aba3bdd-a8ed-4dc3-8571-5ea3dff0a2e0 req-e8101c9d-a67f-4ffe-8031-a35667532b29 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0900123 May 24 10:01:15.653010 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8aba3bdd-a8ed-4dc3-8571-5ea3dff0a2e0 req-3174ebad-37fa-4bdb-9f06-3af235af6cde tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0441210 May 24 10:01:15.853004 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f9afc15-00af-4535-8b36-e2588b2891ed req-aee42a6e-d96a-4840-9681-97e6cfb0cbd5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:15.854494 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7f9afc15-00af-4535-8b36-e2588b2891ed req-aee42a6e-d96a-4840-9681-97e6cfb0cbd5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0857468 May 24 10:01:15.863631 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52512) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:15.869356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'floatingip': {'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:15.951589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.001293 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72c89e04-1b25-478f-ae33-bc3fffd1d970 req-32fe7d51-648f-4e8e-a027-29f51e08ee27 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:16.011716 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72c89e04-1b25-478f-ae33-bc3fffd1d970 req-32fe7d51-648f-4e8e-a027-29f51e08ee27 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1195807 May 24 10:01:16.063384 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72c89e04-1b25-478f-ae33-bc3fffd1d970 req-62599aed-b9f1-43b6-ad6a-cfc446262ccf tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0503833 May 24 10:01:16.327320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52530) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:16.521646 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52542) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:16.612066 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1695322671-network', 'tenant_id': 'bff351576a2f448e94b685ac4edad7c6'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:16.619965 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:16.625642 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Usage tracker for resource:network and tenant:bff351576a2f448e94b685ac4edad7c6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:16.628897 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:16.636036 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:16.702842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Made reservation on behalf of bff351576a2f448e94b685ac4edad7c6 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:16.705681 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.737020 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.745709 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.765871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:16.768437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:16.773302 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Usage tracker for resource:security_group_rule and tenant:bff351576a2f448e94b685ac4edad7c6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:16.776993 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Usage tracker for resource:security_group_rule and tenant:c283ad29da5749c6be3e3fdb79be811e is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:16.779785 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:16.782795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:16.791095 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:16.794237 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:16.841966 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.db.resource_extend [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] It took 0.13 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 10:01:16.940381 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.947674 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:16.949597 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.061637 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.061848 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.068717 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c17b9cbb-1ff6-441f-a4b8-a7ff4357d37e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:17.075367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.119932 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=55ac4200-4e10-4ccd-ad59-18037a438abc,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:17.123599 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 92} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:01:17.128101 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:17.128101 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.046s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:17.135023 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c283ad29da5749c6be3e3fdb79be811e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6114824 May 24 10:01:17.140245 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52554) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:17.154769 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.155106 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Added segment 13331d89-de17-4f02-8e32-cd6b8ebcebba of type vlan for network aaad5285-e76e-499e-a2eb-1c9687521d4c May 24 10:01:17.167154 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1c7ee109-b9fd-4acc-9511-fd28fd24333d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:17.171322 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:17.208101 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.325660 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.327074 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:17.327657 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.333805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:17.372697 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.380340 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Floating IP 29cfb1c8-436d-4666-a731-b479717e97c6 associated. External IP: 172.27.0.38, port: 6fec3696-3d1d-4762-9674-fe87fd37e211. May 24 10:01:17.440177 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Pushing event updated for resources: {'Network': ['ID=aaad5285-e76e-499e-a2eb-1c9687521d4c,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:17.442924 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:17.451339 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network aaad5285-e76e-499e-a2eb-1c9687521d4c details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:01:17.492870 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.540576 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ca65eb3-140f-4aee-8ad8-c0973dcdc55f req-a88afee6-5e71-44cf-89b6-f78f3995342e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:17.548044 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ca65eb3-140f-4aee-8ad8-c0973dcdc55f req-a88afee6-5e71-44cf-89b6-f78f3995342e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.3547499 May 24 10:01:17.555203 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.2267992 May 24 10:01:17.558781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52566) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:17.569797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Request body: {'subnet': {'network_id': 'aaad5285-e76e-499e-a2eb-1c9687521d4c', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1695322671-subnet', 'tenant_id': 'bff351576a2f448e94b685ac4edad7c6', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:17.583509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:17.588448 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Usage tracker for resource:subnet and tenant:bff351576a2f448e94b685ac4edad7c6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:17.592583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:17.608656 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:17.694001 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ca65eb3-140f-4aee-8ad8-c0973dcdc55f req-dfc09d17-998c-4b6f-84a6-6a8bb7beeda6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1450326 May 24 10:01:17.696314 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Made reservation on behalf of bff351576a2f448e94b685ac4edad7c6 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:17.699298 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:17.883680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:01:17.945682 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83fe9a3f-9c37-412f-96e0-77f655a1d35a req-f9cf423c-0f2e-4c36-a910-ad155154b388 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:17.947722 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-83fe9a3f-9c37-412f-96e0-77f655a1d35a req-f9cf423c-0f2e-4c36-a910-ad155154b388 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1042211 May 24 10:01:17.957164 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52578) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:17.977299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'floatingip': {'port_id': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:18.045856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.059281 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-46f09ae9-0dff-4b15-abda-faca3118eca0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:18.079312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=55ac4200-4e10-4ccd-ad59-18037a438abc,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:18.084791 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.141265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:18.388292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.389571 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/55ac4200-4e10-4ccd-ad59-18037a438abc HTTP/1.1" status: 204 len: 168 time: 1.2463751 May 24 10:01:18.413259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.421078 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:18.458891 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:01:18.471337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.474310 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:01:18.493137 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:18.494325 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:18.532167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Pushing event updated for resources: {'Subnet': ['ID=bb5b8904-5b0e-454a-810f-cd13906648c5,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:18.545776 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:18.553736 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Checking segment: {'id': '13331d89-de17-4f02-8e32-cd6b8ebcebba', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': 'aaad5285-e76e-499e-a2eb-1c9687521d4c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:18.554237 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Network hostable DHCP agents. Network: aaad5285-e76e-499e-a2eb-1c9687521d4c, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:01:18.703562 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Network aaad5285-e76e-499e-a2eb-1c9687521d4c is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:01:18.749842 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:18.828523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, aaad5285-e76e-499e-a2eb-1c9687521d4c) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:18.832982 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:18.833565 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:18.841199 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-84238ae4-9c79-4857-b2b9-53bdc75ec341 None None] Network aaad5285-e76e-499e-a2eb-1c9687521d4c requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:18.949527 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.3895130 May 24 10:01:18.964693 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-07a4445d-6443-4640-a9a9-29430334ed52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "PUT /v2.0/floatingips/29cfb1c8-436d-4666-a731-b479717e97c6 HTTP/1.1" status: 200 len: 991 time: 3.1000073 May 24 10:01:18.982843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:19.165875 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0184bd81-8a8f-421b-a7f5-ebb2fd687f64 None None] Network aaad5285-e76e-499e-a2eb-1c9687521d4c requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:19.200896 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '29cfb1c8-436d-4666-a731-b479717e97c6', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.38', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:47Z', 'updated_at': '2021-05-24T10:01:16Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:19.205245 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:01:19.213223 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'status': 'completed', 'code': 200} May 24 10:01:19.219193 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:19.219193 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:19.219522 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52618) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:19.256120 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.285365 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:19.294903 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Usage tracker for resource:security_group_rule and tenant:62f27d1f7f844835abc2ad6bd7529a59 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:19.301129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:19.313023 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:19.369902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7733b9a-d093-4707-a414-79ab821804c3 req-2d0aaaa0-8f57-43cb-ae5b-20505e3e6268 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:19.371403 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-2d0aaaa0-8f57-43cb-ae5b-20505e3e6268 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1099 time: 0.1603546 May 24 10:01:19.478149 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.531167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.548704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-155575a4-b3a3-4edf-8553-74d47b859565 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:19.562724 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.566186 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:19.595829 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=62f27d1f7f844835abc2ad6bd7529a59&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3755159 May 24 10:01:19.596477 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52628) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:19.597931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=07fca366-d2f2-424e-bdae-9a8f4c2e3d6a,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:19.601146 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:19.660189 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.687411 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.703415 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:19.963546 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Floating IP c963c3b2-33f2-42e6-89bf-8d9a8954d2af associated. External IP: 172.27.0.64, port: b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f. May 24 10:01:20.009591 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '29cfb1c8-436d-4666-a731-b479717e97c6', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.38', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:47Z', 'updated_at': '2021-05-24T10:01:16Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:20.013448 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45d357e7-ce35-4cf6-97f2-87222c1fae6c None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '29cfb1c8-436d-4666-a731-b479717e97c6', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.38', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:47Z', 'updated_at': '2021-05-24T10:01:16Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:01:20.079702 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.166614 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.174010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:20.189716 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=07fca366-d2f2-424e-bdae-9a8f4c2e3d6a,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:20.195104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:20.201250 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.202380 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/07fca366-d2f2-424e-bdae-9a8f4c2e3d6a HTTP/1.1" status: 204 len: 168 time: 0.6038320 May 24 10:01:20.238307 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52658) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:20.245372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1648403133-network', 'tenant_id': 'c6ba64b66a6b46dfb69d5a4d7716d603'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:20.250579 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:20.254896 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Usage tracker for resource:network and tenant:c6ba64b66a6b46dfb69d5a4d7716d603 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:20.258614 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:20.268269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:20.337295 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Made reservation on behalf of c6ba64b66a6b46dfb69d5a4d7716d603 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:20.340516 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.377092 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac8f5745-bc88-4dd7-883e-b79658debceb None None] New status for floating IP 29cfb1c8-436d-4666-a731-b479717e97c6: ACTIVE {{(pid=109114) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:01:20.387402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.424022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:20.442534 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Usage tracker for resource:security_group_rule and tenant:c6ba64b66a6b46dfb69d5a4d7716d603 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:20.451036 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:01:20.453133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:20.460137 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:20.570003 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.674720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.679551 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:20.709731 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e5697d5f-1df8-4efb-b52b-67db58817699,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:20.714479 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:20.723564 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 73} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:01:20.736358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.736861 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Added segment 7d29ee2d-3e26-4b2e-8686-94d313ea291b of type vlan for network da034945-4145-49ad-82fb-8d1318c66df8 May 24 10:01:20.766252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.898920 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:01:20.904957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.920352 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.922188 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:20.922836 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:01:20.923301 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:20.929320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9f0eb8c5-b795-4295-bf5b-9bf61af8f231 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:20.937612 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:20.938166 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:21.003837 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8d79727c-38d7-4465-b00c-d891c4e942d2 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "PUT /v2.0/floatingips/c963c3b2-33f2-42e6-89bf-8d9a8954d2af HTTP/1.1" status: 200 len: 990 time: 3.0456462 May 24 10:01:21.005445 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7733b9a-d093-4707-a414-79ab821804c3 req-f4d76882-4c54-4084-b8b5-5bb104851282 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:21.009921 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-f4d76882-4c54-4084-b8b5-5bb104851282 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.2669840 May 24 10:01:21.164587 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Pushing event updated for resources: {'Network': ['ID=da034945-4145-49ad-82fb-8d1318c66df8,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:21.166887 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:21.168701 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.239s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:21.177133 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network da034945-4145-49ad-82fb-8d1318c66df8 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:01:21.214619 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:21.260161 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.0177236 May 24 10:01:21.260723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52694) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:21.266183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Request body: {'subnet': {'network_id': 'da034945-4145-49ad-82fb-8d1318c66df8', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1648403133-subnet', 'tenant_id': 'c6ba64b66a6b46dfb69d5a4d7716d603', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:21.270959 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:21.275980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Usage tracker for resource:subnet and tenant:c6ba64b66a6b46dfb69d5a4d7716d603 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:21.279096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:21.291015 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:21.354493 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Made reservation on behalf of c6ba64b66a6b46dfb69d5a4d7716d603 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:21.357276 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:21.452587 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:01:21.572449 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': 'c963c3b2-33f2-42e6-89bf-8d9a8954d2af', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 142, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:0b:f6:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:51Z', 'updated_at': '2021-05-24T10:01:19Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:21.578032 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:01:21.588090 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:21.588961 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:21.768813 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:21.864257 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:21.869399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a9e43b61-ca16-4354-974f-532c588b2e8c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:21.941099 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Pushing event updated for resources: {'Subnet': ['ID=45122598-0dab-4a6c-8a84-82955688c8b1,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:21.944481 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:21.951489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Checking segment: {'id': '7d29ee2d-3e26-4b2e-8686-94d313ea291b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 73, 'network_id': 'da034945-4145-49ad-82fb-8d1318c66df8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:21.951489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Network hostable DHCP agents. Network: da034945-4145-49ad-82fb-8d1318c66df8, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:01:22.051984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Network da034945-4145-49ad-82fb-8d1318c66df8 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:01:22.136925 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, da034945-4145-49ad-82fb-8d1318c66df8) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:22.141255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:22.142409 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:22.146250 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3ceed986-d90a-4233-97dc-62305b00e880 None None] Network da034945-4145-49ad-82fb-8d1318c66df8 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:22.188636 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': 'c963c3b2-33f2-42e6-89bf-8d9a8954d2af', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 142, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:0b:f6:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:51Z', 'updated_at': '2021-05-24T10:01:19Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:22.193030 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-39ad2604-4842-4777-995b-2720c0fc5f65 None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': 'c963c3b2-33f2-42e6-89bf-8d9a8954d2af', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 142, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:0b:f6:e6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:00:51Z', 'updated_at': '2021-05-24T10:01:19Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:01:22.214011 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 0.9504750 May 24 10:01:22.394322 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-86addc7e-4927-4cf0-8a19-8d03843b1264 None None] Network da034945-4145-49ad-82fb-8d1318c66df8 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:22.479800 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b93de359-d7f3-4c13-b8ac-488103eced85 None None] New status for floating IP c963c3b2-33f2-42e6-89bf-8d9a8954d2af: ACTIVE {{(pid=109114) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:01:22.755412 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-53fb91d7-91d2-4cad-9a8d-004c5fea1d37 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 1001 time: 0.1185603 May 24 10:01:22.757277 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'status': 'completed', 'code': 200} May 24 10:01:22.865638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7733b9a-d093-4707-a414-79ab821804c3 req-c91c926b-281c-434d-8d94-d312726aa5bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:22.867217 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-c91c926b-281c-434d-8d94-d312726aa5bf service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0952611 May 24 10:01:22.873394 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-8916c772-5db1-4af5-9ca4-a45df29dccb0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:22.879540 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-8916c772-5db1-4af5-9ca4-a45df29dccb0 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.0966229 May 24 10:01:22.966531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7733b9a-d093-4707-a414-79ab821804c3 req-61d45c67-db14-4c65-9433-a1865c4ae8c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:22.968379 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-61d45c67-db14-4c65-9433-a1865c4ae8c8 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0927055 May 24 10:01:23.022609 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-9e1a7173-1c35-40ac-a208-37af31fc35e0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:23.024054 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-9e1a7173-1c35-40ac-a208-37af31fc35e0 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.1359346 May 24 10:01:23.109947 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-d5aa1dd8-a920-45fe-965c-050fd0d6dc7a service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1346858 May 24 10:01:23.136253 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-ac4e70b3-1093-406e-adbf-9546b17a5570 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1000 time: 0.0974393 May 24 10:01:23.260291 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a7733b9a-d093-4707-a414-79ab821804c3 req-e07b0bca-43cd-41d8-8749-97ec40a3386c service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1436899 May 24 10:01:23.262323 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-106e565f-3c6d-4b66-9b65-c7f0088c1808 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:23.263825 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-106e565f-3c6d-4b66-9b65-c7f0088c1808 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.1211815 May 24 10:01:23.361394 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-0bfbb45a-79f7-4992-af07-9f1813ff7f10 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:23.362887 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-0bfbb45a-79f7-4992-af07-9f1813ff7f10 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0777996 May 24 10:01:23.512656 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-f1c519ad-ae9e-412e-995d-d2f21657f4d5 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1379032 May 24 10:01:23.662185 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0aafcbc-cf31-40a0-b03e-f6bf4a435f23 req-05a49b2d-bacc-405b-8ee1-8ff00d06f95b service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1419880 May 24 10:01:24.808372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52792) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:24.967636 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:24.984076 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:24.988331 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Usage tracker for resource:security_group_rule and tenant:b576bd59c1ab439eaa4a5c086392ce63 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:24.991566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:24.997682 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:25.089127 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.170872 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.180364 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:25.225938 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b576bd59c1ab439eaa4a5c086392ce63&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4140975 May 24 10:01:25.226310 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52814) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:25.229690 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=48480d93-e658-4b10-91ca-91157fd394b2,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:25.250896 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.251256 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.251608 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.251952 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.252291 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.252647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.253019 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.253320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.253658 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.253988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.254318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.254647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:25.261127 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:25.310758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.342113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.474130 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.478409 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-304fb6e7-6b3e-4f43-9627-1e9077ee68a1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:25.491120 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=48480d93-e658-4b10-91ca-91157fd394b2,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:25.493329 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:25.497504 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:25.498642 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/48480d93-e658-4b10-91ca-91157fd394b2 HTTP/1.1" status: 204 len: 168 time: 0.2713010 May 24 10:01:26.109513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52848) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:26.127509 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.142586 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:26.146488 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Usage tracker for resource:security_group_rule and tenant:8e7a880f846d427b831bbc938f496462 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:26.149453 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:26.156596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:26.262519 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.353890 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.363541 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:26.409775 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=8e7a880f846d427b831bbc938f496462&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2986200 May 24 10:01:26.410675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cdc97deb-2454-4553-89ea-a16c581d830a,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:26.412619 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52860) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:26.418976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.419338 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.419686 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.420031 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.421846 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.422199 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.422544 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.422889 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.423231 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.423564 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.423894 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.424223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:26.437946 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:26.476166 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.501808 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.645834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.651197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f72361ff-4f69-4ee3-8e95-b7ac8240fa83 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:26.664342 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cdc97deb-2454-4553-89ea-a16c581d830a,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:26.667507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:26.669353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:26.677873 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/cdc97deb-2454-4553-89ea-a16c581d830a HTTP/1.1" status: 204 len: 168 time: 0.2562852 May 24 10:01:26.932094 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49879f95-599e-4534-9f64-2683d6e2efbd req-28abbfbd-2224-4c45-b4e4-cb9631a1e5c5 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:26.933778 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-49879f95-599e-4534-9f64-2683d6e2efbd req-28abbfbd-2224-4c45-b4e4-cb9631a1e5c5 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0848434 May 24 10:01:26.979783 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-49879f95-599e-4534-9f64-2683d6e2efbd req-10088f85-66b1-4950-8a9e-3b40e6e71bed tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403075 May 24 10:01:27.395258 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 52900) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:27.414104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.429473 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:27.433709 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Usage tracker for resource:security_group_rule and tenant:ca6bb15d0c914eb9ac7aafbec1828559 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:27.436856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:27.442916 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:27.531505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.576642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.584006 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cc4de9a0-5e10-4dfa-94a2-1ef29f94f958 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:27.625407 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ca6bb15d0c914eb9ac7aafbec1828559&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2288511 May 24 10:01:27.626270 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ae1616fd-ff37-4f1e-b871-b99a13b75b79,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:27.628022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52906) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:27.630202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:27.692010 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.718904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.844782 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.848557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:27.859947 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ae1616fd-ff37-4f1e-b871-b99a13b75b79,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:27.863941 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:27.867272 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:27.868917 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/ae1616fd-ff37-4f1e-b871-b99a13b75b79 HTTP/1.1" status: 204 len: 168 time: 0.2387633 May 24 10:01:28.980049 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52946) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:29.084209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-409744915-network', 'tenant_id': '6f38795dc42f4d5186770d3b222547ae'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:29.089320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:29.093067 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Usage tracker for resource:network and tenant:6f38795dc42f4d5186770d3b222547ae is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:29.095900 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:29.101979 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:29.181943 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Made reservation on behalf of 6f38795dc42f4d5186770d3b222547ae for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:29.184229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.196171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.210408 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:29.214931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Usage tracker for resource:security_group_rule and tenant:6f38795dc42f4d5186770d3b222547ae is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:29.218043 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:29.224062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:29.302428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.365261 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.368371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b19c3db3-9c62-421c-acd1-c0e0df48fa09 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:29.397151 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 82} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:01:29.407333 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d467c95d-fd2f-4e55-82e5-b5b76f6881e7,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:29.407333 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:29.411699 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.412165 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Added segment 0af48d33-b0f2-492b-b676-993923c0f33a of type vlan for network 570c27de-a372-4133-bc81-83e9d6aa9b15 May 24 10:01:29.445675 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.519450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.520768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:29.521366 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:29.526179 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:29.621121 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Pushing event updated for resources: {'Network': ['ID=570c27de-a372-4133-bc81-83e9d6aa9b15,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:29.624040 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:29.633098 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:01:29.652088 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 880 time: 0.6709583 May 24 10:01:29.654986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 52958) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:29.661263 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Request body: {'subnet': {'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-409744915-subnet', 'tenant_id': '6f38795dc42f4d5186770d3b222547ae', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:29.669143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:29.673015 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Usage tracker for resource:subnet and tenant:6f38795dc42f4d5186770d3b222547ae is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:29.676314 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:29.683244 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:29.753118 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Made reservation on behalf of 6f38795dc42f4d5186770d3b222547ae for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:29.755465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:30.186107 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:30.190407 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:30.260696 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Pushing event updated for resources: {'Subnet': ['ID=1f8e051e-bd56-4915-8e94-622f6ff108d3,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:30.264438 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:30.268597 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Checking segment: {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:30.269151 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Network hostable DHCP agents. Network: 570c27de-a372-4133-bc81-83e9d6aa9b15, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:01:30.359781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:01:30.445631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 570c27de-a372-4133-bc81-83e9d6aa9b15) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:30.450528 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:30.451303 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:30.452943 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1634c14e-140e-43bc-b9d0-93c6d21a2cea req-aa240aaa-8275-4a19-9fd6-9a77eea51457 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:30.457554 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f6e2b5c6-704b-49c0-982f-c32552a7a274 None None] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:30.457871 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1634c14e-140e-43bc-b9d0-93c6d21a2cea req-aa240aaa-8275-4a19-9fd6-9a77eea51457 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0874591 May 24 10:01:30.501447 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1634c14e-140e-43bc-b9d0-93c6d21a2cea req-a29d48e4-f2e1-470c-a7cf-174da189f00e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413926 May 24 10:01:30.594789 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.9384725 May 24 10:01:30.662175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-32009699-369d-4807-aae3-742d30adbb7f None None] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:33.279220 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-53b98583-9a85-4aa0-8b12-02355a704d79 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:34.145534 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bec3630b-d2e6-4f5f-8e89-a4f49c58f16a req-5b884e84-1393-4636-b69a-7fe1e5e9ef6f tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:34.146963 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bec3630b-d2e6-4f5f-8e89-a4f49c58f16a req-5b884e84-1393-4636-b69a-7fe1e5e9ef6f tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0796950 May 24 10:01:34.193066 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bec3630b-d2e6-4f5f-8e89-a4f49c58f16a req-141c01e1-3a01-4726-a365-e47c9e36e970 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403287 May 24 10:01:35.261810 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-a7ee4f37-1266-449a-8eaf-859a9134724e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:35.263617 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-a7ee4f37-1266-449a-8eaf-859a9134724e tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.95 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.1369116 May 24 10:01:35.398639 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-977f0da4-031b-425d-816d-4f4436632a3a req-952ad0a6-9514-4d72-86c5-41bbf4c33071 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:35.400766 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-977f0da4-031b-425d-816d-4f4436632a3a req-952ad0a6-9514-4d72-86c5-41bbf4c33071 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 945 time: 0.0946510 May 24 10:01:35.431592 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Deleting port 35e9079c-497b-4448-a787-0fdee4f4fc4c {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:01:35.433130 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:35.453266 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-977f0da4-031b-425d-816d-4f4436632a3a req-d39e18f5-7804-4c9c-9291-df54d60e7ec5 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/security-groups?id=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0459807 May 24 10:01:35.599307 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] For port 35e9079c-497b-4448-a787-0fdee4f4fc4c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=35e9079c-497b-4448-a787-0fdee4f4fc4c,segment=NetworkSegment(deba89ad-e4b9-43a6-b2c3-6904c93466bf),segment_id=deba89ad-e4b9-43a6-b2c3-6904c93466bf)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:35.599840 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:35.728349 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Calling delete_port for 35e9079c-497b-4448-a787-0fdee4f4fc4c owned by compute:nova {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:01:35.817089 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-29a282ef-cedf-4f3e-b836-a0db322fe610 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:35.819736 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:35.948513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-754f6400-7f76-4b42-88bd-c77933ac7861 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:35.949247 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2e27324f-39bf-485f-9479-b756e81203d5 req-754f6400-7f76-4b42-88bd-c77933ac7861 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Synchronizing usage tracker for tenant:6f38795dc42f4d5186770d3b222547ae on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:36.056906 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2e27324f-39bf-485f-9479-b756e81203d5 req-754f6400-7f76-4b42-88bd-c77933ac7861 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Unset dirty status for tenant:6f38795dc42f4d5186770d3b222547ae on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:36.058635 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-754f6400-7f76-4b42-88bd-c77933ac7861 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 808 time: 0.3371117 May 24 10:01:36.073787 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-18cb0e04-f911-4673-8fa9-654580c83ada tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/quotas/6f38795dc42f4d5186770d3b222547ae HTTP/1.1" status: 200 len: 375 time: 0.0089567 May 24 10:01:36.124136 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-f9389e7e-ad45-4192-9591-68e37ba590ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0449541 May 24 10:01:36.154098 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:36.171211 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['35e9079c-497b-4448-a787-0fdee4f4fc4c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:01:36.175037 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b27ae57f-3c5f-4091-954e-52ca3ac3cd29 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:36.231140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Pushing event deleted for resources: {'Port': ['ID=35e9079c-497b-4448-a787-0fdee4f4fc4c,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:36.284867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:36.373761 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] No DVR routers for this DVR port 35e9079c-497b-4448-a787-0fdee4f4fc4c on host hv-cinder-79123 {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:01:36.380136 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:36.382723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'name': 'network-vif-deleted', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:36.408684 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-3a285c76-ee51-45f9-86ed-82fffa918acf service neutron] 10.222.0.67 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2776837 May 24 10:01:36.450465 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'af25f39c-cd76-47bd-aace-363ee0770877', 'name': 'network-vif-deleted', 'tag': '35e9079c-497b-4448-a787-0fdee4f4fc4c', 'status': 'completed', 'code': 200} May 24 10:01:36.508046 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c205810f-e0c9-4200-b2d2-5837f6d357d4 req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.95 "DELETE /v2.0/ports/35e9079c-497b-4448-a787-0fdee4f4fc4c HTTP/1.1" status: 204 len: 173 time: 1.2380321 May 24 10:01:36.550158 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9ff451a3-f8a4-4422-9eb3-30106f3bc772 service neutron] 10.222.0.67 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1352143 May 24 10:01:36.629661 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e53ae4bb-8633-462d-8963-f47f1f699164 req-1269629b-c279-4d24-836b-679647f1ffdb tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.0465758 May 24 10:01:36.884973 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-98431955-68ce-405f-aa2f-607e1a8e02e4 req-65c64322-abbd-477a-9508-f6b371e9bea1 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.0469384 May 24 10:01:37.222954 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:37.317805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53140) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:37.501512 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:37.519604 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:37.523481 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Usage tracker for resource:security_group_rule and tenant:33ec3f74996b45d1a901fa793079166e is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:37.527050 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:37.533256 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:37.658299 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:37.736797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:37.747111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:37.822193 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=33ec3f74996b45d1a901fa793079166e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5031114 May 24 10:01:37.826185 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53162) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:37.833255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4fbfa98f-f8de-4a06-8b9f-40150e0675b5 req-acea70b7-aa99-4723-91fc-18ec9186d3db tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Synchronizing usage tracker for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:37.835828 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b23901fd-ef46-4462-aa4c-8cdf2bb04123,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:37.839755 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0e1ebe1b-3844-4d3a-82d2-96f7a3ce8377 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:37.931378 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:37.935887 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4fbfa98f-f8de-4a06-8b9f-40150e0675b5 req-acea70b7-aa99-4723-91fc-18ec9186d3db tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Unset dirty status for tenant:35a01011c3994805ba2a956d0daf4aa6 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:37.937345 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4fbfa98f-f8de-4a06-8b9f-40150e0675b5 req-acea70b7-aa99-4723-91fc-18ec9186d3db tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] 10.222.0.67 "GET /v2.0/ports?device_id=af25f39c-cd76-47bd-aace-363ee0770877 HTTP/1.1" status: 200 len: 210 time: 0.1664338 May 24 10:01:37.964927 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.011699 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53170) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:38.063624 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-90c1f535-f3c6-4a44-b6dc-3c5b29dae542 req-98fb5a03-fb09-4c3f-a36e-52b6339794b8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.0547328 May 24 10:01:38.136640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.150532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:38.154331 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Usage tracker for resource:security_group_rule and tenant:0f3d50a0241645d395417ddabc565ab2 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:38.157039 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:38.162596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:38.193925 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.199156 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:38.216277 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b23901fd-ef46-4462-aa4c-8cdf2bb04123,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:38.219206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:38.225902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.231847 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b23901fd-ef46-4462-aa4c-8cdf2bb04123 HTTP/1.1" status: 204 len: 168 time: 0.3997285 May 24 10:01:38.290122 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.359156 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.368091 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c21be406-dab1-4594-a59e-f3737602c4b5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:38.406501 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=0f3d50a0241645d395417ddabc565ab2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3937097 May 24 10:01:38.407464 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d1ceaed6-13d3-415c-bdd1-0f9868780b53,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:38.409256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53182) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:38.412120 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:38.477826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.506934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.610277 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.615619 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d6d53646-76df-4204-9e05-d2705f1a9868 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:38.630509 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d1ceaed6-13d3-415c-bdd1-0f9868780b53,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:38.632586 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:38.633515 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.637303 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/d1ceaed6-13d3-415c-bdd1-0f9868780b53 HTTP/1.1" status: 204 len: 168 time: 0.2243195 May 24 10:01:38.725357 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.886275 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53196) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:38.904069 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:38.918149 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:38.921950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Usage tracker for resource:security_group_rule and tenant:dec273c40e864162b8266034a4dac612 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:38.924785 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:38.930633 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:39.016440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.090502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.173437 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:39.211589 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 35e9079c-497b-4448-a787-0fdee4f4fc4c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:01:39.212794 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=dec273c40e864162b8266034a4dac612&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3253965 May 24 10:01:39.214508 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7456a9e3-1f55-422a-bf86-e2f5f867162d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:39.215497 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53206) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:39.217805 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:39.245951 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 35e9079c-497b-4448-a787-0fdee4f4fc4c {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:01:39.248460 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 35e9079c-497b-4448-a787-0fdee4f4fc4c not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:01:39.281029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.372149 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.491413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53212) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:39.508339 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.512884 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:39.518960 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.538628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7456a9e3-1f55-422a-bf86-e2f5f867162d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:39.542163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:39.544869 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:39.551254 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Usage tracker for resource:security_group_rule and tenant:f68e3167abbe48f2980025a39a7ea8a6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:39.555777 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:39.566866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:39.566866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.567464 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/7456a9e3-1f55-422a-bf86-e2f5f867162d HTTP/1.1" status: 204 len: 168 time: 0.3484716 May 24 10:01:39.648537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.724180 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.733614 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-969db129-af1c-4635-bf3d-cfefe49e0223 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:39.772128 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=f68e3167abbe48f2980025a39a7ea8a6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2796590 May 24 10:01:39.773749 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f2747758-016b-4e02-9432-05fc62a09467,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:39.774940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53220) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:39.777577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:39.842467 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.868718 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.965201 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.970135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d531ffee-7997-4d54-9820-fb33343ac723 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:39.983735 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f2747758-016b-4e02-9432-05fc62a09467,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:39.990030 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:39.991235 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:39.992331 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/f2747758-016b-4e02-9432-05fc62a09467 HTTP/1.1" status: 204 len: 168 time: 0.2152944 May 24 10:01:41.593834 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b3e796f8-88cb-430d-8d18-d3065395036f req-2b073c67-a862-4e89-b4e3-8474c9ff792e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.0575380 May 24 10:01:42.478125 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-2165789b-f60b-4359-8d19-baf2b4049098 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:42.480837 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-2165789b-f60b-4359-8d19-baf2b4049098 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 808 time: 0.1821799 May 24 10:01:42.504024 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Request body: {'port': {'device_id': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15', 'admin_state_up': True, 'tenant_id': '6f38795dc42f4d5186770d3b222547ae'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:42.719455 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:42.723694 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Usage tracker for resource:port and tenant:6f38795dc42f4d5186770d3b222547ae is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:42.727424 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Quota usage for port was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:42.734700 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:42.788427 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bd5b10a1-d23f-40db-b5f1-bb3bcbd48a43 req-9ab6acf3-b3f1-421f-a0ad-e27e675ec2df tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.0548084 May 24 10:01:42.839688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Made reservation on behalf of 6f38795dc42f4d5186770d3b222547ae for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:42.844083 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.023353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Allocated IP 172.20.0.3 (570c27de-a372-4133-bc81-83e9d6aa9b15/1f8e051e-bd56-4915-8e94-622f6ff108d3/a98f21e3-a592-42a4-b016-d2cf72a32e5c) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:01:43.248441 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.428074 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.435991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:43.511143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:01:43.512054 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:43.514583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:01:43.520038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:43.520651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.556268 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Pushing event updated for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:43.565988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:43.611672 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1180172 May 24 10:01:43.627778 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Request body: {'port': {'device_id': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:43.763745 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.788500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53396) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:43.873220 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Deleting subnet b28ebd86-e35c-4b85-b6ba-3336770f64e8 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:01:43.876179 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.964721 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:43.984492 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.100127 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d47a07f6-961b-49c0-ac00-97686d5a7bcb req-a264f7cb-6f47-44a8-9645-16fd33276921 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:44.101674 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d47a07f6-961b-49c0-ac00-97686d5a7bcb req-a264f7cb-6f47-44a8-9645-16fd33276921 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 931 time: 0.1767147 May 24 10:01:44.146692 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a98f21e3-a592-42a4-b016-d2cf72a32e5c {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:44.154927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.171613 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Transition to ACTIVE for port object a98f21e3-a592-42a4-b016-d2cf72a32e5c will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:01:44.234944 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d47a07f6-961b-49c0-ac00-97686d5a7bcb req-4134e8ff-5d48-4c2e-be8a-502e2ff403de tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1229746 May 24 10:01:44.273647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.278542 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:44.407770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.422881 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f389072e-18b7-44af-a059-2c88b1e937e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:44.434200 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Pushing event updated for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:44.441801 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:44.462174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Attempting to bind port a98f21e3-a592-42a4-b016-d2cf72a32e5c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:01:44.462174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Attempting to bind port a98f21e3-a592-42a4-b016-d2cf72a32e5c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:01:44.462174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Attempting to bind port a98f21e3-a592-42a4-b016-d2cf72a32e5c on network 570c27de-a372-4133-bc81-83e9d6aa9b15 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:44.465206 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Port a98f21e3-a592-42a4-b016-d2cf72a32e5c on network 570c27de-a372-4133-bc81-83e9d6aa9b15 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:01:44.466326 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Attempting to bind port a98f21e3-a592-42a4-b016-d2cf72a32e5c on network 570c27de-a372-4133-bc81-83e9d6aa9b15 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:44.475353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:01:44.475708 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Checking segment: {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:44.476100 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Bound using segment: {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:01:44.476537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Bound port: a98f21e3-a592-42a4-b016-d2cf72a32e5c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:01:44.476885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.501125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.553379 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=b28ebd86-e35c-4b85-b6ba-3336770f64e8,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:44.564944 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/b28ebd86-e35c-4b85-b6ba-3336770f64e8 HTTP/1.1" status: 204 len: 168 time: 0.7730961 May 24 10:01:44.570518 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:44.575376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53420) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:44.644219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:01:44.647870 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:01:44.697662 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.700223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.702750 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:44.711872 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-535b4115-3113-495e-8a87-2f175225761a service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2063792 May 24 10:01:44.781410 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Pushing event updated for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:44.786009 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:44.790252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:44.790867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.793168 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] 10.222.0.95 "PUT /v2.0/ports/a98f21e3-a592-42a4-b016-d2cf72a32e5c HTTP/1.1" status: 200 len: 1093 time: 1.1719055 May 24 10:01:44.793664 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:44.875011 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:44.894028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > segment before_delete 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.894695 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.895634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.896103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > port after_create 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.896672 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > port after_delete 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.897219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > port after_update 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.897731 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > segment after_delete 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.898442 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.898944 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.899426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet after_update 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:44.969946 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c', 'status': 'completed', 'code': 200} May 24 10:01:44.973182 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-003ff4e3-3dff-4cc4-ba60-39815cc709a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:44.974791 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-003ff4e3-3dff-4cc4-ba60-39815cc709a4 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1099 time: 0.1164165 May 24 10:01:45.071009 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:45.071638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:45.072242 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet before_create 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:45.073240 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Subscribe: > subnet before_update 55550000 {{(pid=109113) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:01:45.073867 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['1c6bd2e7-cf1c-4cda-aa53-7f057dec1138']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:45.143857 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-6d392071-7422-4627-90f5-d76446d2ea63 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.145736 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-6d392071-7422-4627-90f5-d76446d2ea63 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.1484559 May 24 10:01:45.149005 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'deba89ad-e4b9-43a6-b2c3-6904c93466bf') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:45.149337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'deba89ad-e4b9-43a6-b2c3-6904c93466bf') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:45.175752 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:45.204157 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:45.231448 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Releasing vlan 55 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:01:45.232104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: from cryptography.utils import int_from_bytes May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: from cryptography.utils import int_from_bytes May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:01:45.252253 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:01:45.280427 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-c98e0a1f-73cf-4bfe-95b2-fd9f407a6257 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 1001 time: 0.1268220 May 24 10:01:45.335880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-b6eab381-aa94-40ce-b172-04e338ac18ce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.338030 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-b6eab381-aa94-40ce-b172-04e338ac18ce service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 1096 time: 0.2543356 May 24 10:01:45.375471 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-0622a2b4-ea6f-4c21-a087-ee2ebec48010 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.376916 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-0622a2b4-ea6f-4c21-a087-ee2ebec48010 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0879810 May 24 10:01:45.421659 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-f0235e27-1739-49cd-810e-22f2c2c755ba service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c HTTP/1.1" status: 200 len: 217 time: 0.0667953 May 24 10:01:45.424395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:45.484644 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-f0cc597d-01cb-4e68-9cc4-9dd4f1162fda service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.486229 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-f0cc597d-01cb-4e68-9cc4-9dd4f1162fda service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0979686 May 24 10:01:45.516465 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-648fb089-6f79-457d-aec8-ab9175ff6d4e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:45.542034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:45.561299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Pushing event deleted for resources: {'Network': ['ID=1c6bd2e7-cf1c-4cda-aa53-7f057dec1138,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:45.570712 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:45.576973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e27324f-39bf-485f-9479-b756e81203d5 req-a6ba8e1b-c66f-47bb-b073-3d0667ffccb1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.578437 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-a6ba8e1b-c66f-47bb-b073-3d0667ffccb1 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=1f8e051e-bd56-4915-8e94-622f6ff108d3 HTTP/1.1" status: 200 len: 848 time: 0.1190896 May 24 10:01:45.630389 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3866d9fc-0f94-4bd7-b0f5-7f141efcac93 req-e5c07817-ae38-4b35-9c84-e6cde8719225 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:45.637962 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3866d9fc-0f94-4bd7-b0f5-7f141efcac93 req-e5c07817-ae38-4b35-9c84-e6cde8719225 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 943 time: 0.2149620 May 24 10:01:45.662215 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-73ab7953-0a3c-468d-8464-7537f7ea5180 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=570c27de-a372-4133-bc81-83e9d6aa9b15&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0725462 May 24 10:01:45.668359 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] 10.222.0.67 "DELETE /v2.0/networks/1c6bd2e7-cf1c-4cda-aa53-7f057dec1138 HTTP/1.1" status: 204 len: 168 time: 1.0916345 May 24 10:01:45.686617 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-492ff92d-9adc-4fb4-a00b-95124ac7fde5 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1909673 May 24 10:01:45.818659 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3866d9fc-0f94-4bd7-b0f5-7f141efcac93 req-bb6b8e87-67d0-4c81-8d57-249d6afa0dd8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1798396 May 24 10:01:45.878590 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85a59cda-9e04-4f8d-b3ad-d52e0d246d71 req-7fa2f88f-7387-410f-a32d-092ac43ebc18 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1829019 May 24 10:01:45.917517 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-d45a931b-c3b4-4576-8d48-96b2b414707d service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2468648 May 24 10:01:46.105495 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e27324f-39bf-485f-9479-b756e81203d5 req-537ed803-8a48-4db9-9930-d512b634e2c7 service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1797559 May 24 10:01:46.241505 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53478) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:46.293428 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6c7fc38c-b938-4d00-a82e-1a210087d6b5 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=35a01011c3994805ba2a956d0daf4aa6&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0503361 May 24 10:01:46.295860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53482) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:46.310073 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-3f70dcde-0156-4c08-b3bc-4648cc28c535 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:46.311669 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-3f70dcde-0156-4c08-b3bc-4648cc28c535 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 1096 time: 0.0772429 May 24 10:01:46.361504 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:46.388630 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:46.469793 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-43060bb3-2ca9-4685-9ce5-70d32d2cdfd2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:46.476558 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-43060bb3-2ca9-4685-9ce5-70d32d2cdfd2 service neutron] 10.222.0.95 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 908 time: 0.1526735 May 24 10:01:46.536859 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:46.546633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-aa719434-005e-45f4-aed5-72024cb6df7f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:46.550693 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-79610590-b314-43ca-a84c-c86135c506cc service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c HTTP/1.1" status: 200 len: 217 time: 0.0680907 May 24 10:01:46.558706 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=329ebe57-82df-4cee-a5e2-a2d55e0f5d7b,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:46.567133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:46.630934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-958df58f-29f3-4577-b07a-ca9139fd5cad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:46.632432 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-958df58f-29f3-4577-b07a-ca9139fd5cad service neutron] 10.222.0.95 "GET /v2.0/subnets?id=1f8e051e-bd56-4915-8e94-622f6ff108d3 HTTP/1.1" status: 200 len: 848 time: 0.0735338 May 24 10:01:46.644125 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:46.645265 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/329ebe57-82df-4cee-a5e2-a2d55e0f5d7b HTTP/1.1" status: 204 len: 168 time: 0.3484788 May 24 10:01:46.691996 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-2ab51736-cc0f-4e64-8c37-526aa8ea45ee service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=570c27de-a372-4133-bc81-83e9d6aa9b15&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0502875 May 24 10:01:46.831067 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-80e8b8ff-4510-4d2b-8242-e57fc50bbf59 service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1314480 May 24 10:01:46.978618 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edc20771-0bed-4498-8dc4-cdf185b31995 req-2c0ac0cc-2470-4b8b-a445-0a88d04a9dfc service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1389625 May 24 10:01:47.050242 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-0e0542fb-5897-4b42-b1b2-785726466bc8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:47.051092 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-0e0542fb-5897-4b42-b1b2-785726466bc8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Synchronizing usage tracker for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:47.165242 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-0e0542fb-5897-4b42-b1b2-785726466bc8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Unset dirty status for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:47.172048 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-0e0542fb-5897-4b42-b1b2-785726466bc8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 943 time: 0.2155836 May 24 10:01:47.274128 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-45882df3-3401-480f-839d-7db0762751c4 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Synchronizing usage tracker for tenant:6f38795dc42f4d5186770d3b222547ae on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:47.359489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-45882df3-3401-480f-839d-7db0762751c4 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Unset dirty status for tenant:6f38795dc42f4d5186770d3b222547ae on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:47.360761 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53643a69-1955-4dc6-beed-42719eeeb8d1 req-45882df3-3401-480f-839d-7db0762751c4 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1808884 May 24 10:01:47.799374 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53554) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:47.953600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Request body: {'network': {'name': 'tempest-ServerRescueNegativeTestJSON-786240911-network', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:47.958977 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:47.962938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Usage tracker for resource:network and tenant:b08409afad374227ace1cdf31e4e9412 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:47.966293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:47.972593 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:48.079570 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Made reservation on behalf of b08409afad374227ace1cdf31e4e9412 for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:48.082808 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.098022 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.113398 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:48.117210 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Usage tracker for resource:security_group_rule and tenant:b08409afad374227ace1cdf31e4e9412 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:48.120103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:48.125787 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:48.211650 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.273541 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.277258 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8eca91a9-cdc4-48ff-a057-1d7264c71686 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:48.306103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 71} {{(pid=109113) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:01:48.315068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fe64f46b-8749-477e-bbac-58ca92a691c2,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:48.318551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:48.321494 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.321999 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Added segment bb46500f-61f2-431d-97f0-2ba79eaa0fd2 of type vlan for network b0fff367-5ea3-4833-b1bf-44e4dd284af1 May 24 10:01:48.358279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.478392 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.480270 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:48.480874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.484435 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-449cdafa-cda9-4056-bf27-f1345c656221 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:48.640170 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Pushing event updated for resources: {'Network': ['ID=b0fff367-5ea3-4833-b1bf-44e4dd284af1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:48.643050 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:48.652340 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 890 time: 0.8518839 May 24 10:01:48.652636 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:01:48.656257 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53582) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:48.668341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Request body: {'subnet': {'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'cidr': '172.20.0.0/28', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-subnet', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:48.668341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:48.671404 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Usage tracker for resource:subnet and tenant:b08409afad374227ace1cdf31e4e9412 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:48.672986 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93606556-c0c3-472a-a6ad-5e2bdb755668 req-c47af96d-44f6-4522-8e66-40af6d282a63 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:48.674498 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93606556-c0c3-472a-a6ad-5e2bdb755668 req-c47af96d-44f6-4522-8e66-40af6d282a63 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 943 time: 0.1760643 May 24 10:01:48.681653 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:48.688209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:48.721751 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93606556-c0c3-472a-a6ad-5e2bdb755668 req-664b7f88-4100-4b09-a192-284662736f8a tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0411849 May 24 10:01:48.759068 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Made reservation on behalf of b08409afad374227ace1cdf31e4e9412 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:01:48.762019 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:48.869304 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device a98f21e3-a592-42a4-b016-d2cf72a32e5c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:01:48.981361 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53590) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:49.066037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:49.107691 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c', 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15', 'port_id': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c', 'mac_address': 'fa:16:3e:11:7d:63', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1f8e051e-bd56-4915-8e94-622f6ff108d3', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:01:49.235021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.248044 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53604) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:49.275299 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a98f21e3-a592-42a4-b016-d2cf72a32e5c {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:01:49.297676 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:49.330184 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.351902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:49.353911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.358230 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:49.358526 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Usage tracker for resource:security_group_rule and tenant:b63196e6f2a44b46959c404234303eb1 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:49.361610 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:49.368564 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:49.398083 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.406686 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:49.472994 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.500055 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Pushing event updated for resources: {'Subnet': ['ID=98c67f17-d401-4c79-8169-5da43506ba3c,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:49.508614 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:49.508763 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:49.517405 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.517772 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.518643 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.518770 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.518928 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.519456 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.519794 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.521172 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.527026 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.527434 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.527779 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.528112 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.528596 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.529115 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:49.541584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-17257fa7-6d28-422a-b4cb-8479f6f4bf8e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:49.548914 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.188s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:49.556228 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:49.559183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Checking segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:49.559535 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Network hostable DHCP agents. Network: b0fff367-5ea3-4833-b1bf-44e4dd284af1, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:01:49.564779 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Usage tracker for resource:security_group_rule and tenant:ec4dba65bb4f4ad1ad0833830b1c2a34 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:49.567838 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:49.580404 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:49.604812 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b9df6381-7945-4df8-9a8e-e6af1161594e,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:49.608381 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:49.613883 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b63196e6f2a44b46959c404234303eb1&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3614185 May 24 10:01:49.623054 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53614) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:49.643144 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['a98f21e3-a592-42a4-b016-d2cf72a32e5c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:01:49.795119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:01:49.815222 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.884822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.909484 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.922460 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, b0fff367-5ea3-4833-b1bf-44e4dd284af1) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:49.931567 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6eb2a1a0-f9b2-4e66-89e2-69f61ec7c657 None None] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:49.941580 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:49.942160 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:49.999433 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cede4028-993a-47f8-b5b7-d929925d74e2 req-d51e6cdc-db55-4f7b-9b6c-c64d798ad469 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:50.000875 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cede4028-993a-47f8-b5b7-d929925d74e2 req-d51e6cdc-db55-4f7b-9b6c-c64d798ad469 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 944 time: 0.1561711 May 24 10:01:50.023155 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.033573 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-fbf40617-bb99-40c3-b637-b18d9fc204ae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:50.045870 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.087729 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=eda0893f-47c4-4863-ae0c-ceb91bed9cb9,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:50.098043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53626) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:50.098155 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ec4dba65bb4f4ad1ad0833830b1c2a34&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1156611 May 24 10:01:50.099701 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:50.102546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.109551 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 855 time: 1.4521523 May 24 10:01:50.111251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53630) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:50.113215 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c20d0987-183a-4def-ab57-38897e3b50f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:50.122883 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cede4028-993a-47f8-b5b7-d929925d74e2 req-24263ae0-9267-4c5e-b0eb-479093a9126b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1153700 May 24 10:01:50.129121 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Request body: {'router': {'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:01:50.138033 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:50.143812 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b9df6381-7945-4df8-9a8e-e6af1161594e,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:50.146882 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:50.147872 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Usage tracker for resource:router and tenant:b08409afad374227ace1cdf31e4e9412 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:50.154963 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:50.158570 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.162390 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b9df6381-7945-4df8-9a8e-e6af1161594e HTTP/1.1" status: 204 len: 168 time: 0.5437212 May 24 10:01:50.173168 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:50.202910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5dd28800-60a0-4f4c-a49a-d96b16b89eb7 None None] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:01:50.224928 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.257000 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.258253 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.282134 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.283187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.406595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.412068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f6abd197-8bf5-46e3-be1b-d2b0eb17240b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:50.425436 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=eda0893f-47c4-4863-ae0c-ceb91bed9cb9,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:50.429431 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:50.435760 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.436868 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/eda0893f-47c4-4863-ae0c-ceb91bed9cb9 HTTP/1.1" status: 204 len: 168 time: 0.3377862 May 24 10:01:50.631292 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.884464 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device a98f21e3-a592-42a4-b016-d2cf72a32e5c up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:01:50.928177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53650) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:50.960938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:50.978439 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:50.983077 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Usage tracker for resource:security_group_rule and tenant:d974e0a5b2bf48cfad61f4928a055b27 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:50.985937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:50.991749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:51.050959 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port a98f21e3-a592-42a4-b016-d2cf72a32e5c completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:51.062676 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port a98f21e3-a592-42a4-b016-d2cf72a32e5c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:51.063282 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.078982 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.091451 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.116042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.143512 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.151788 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:51.231904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53660) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:51.253245 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.270721 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.279517 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:51.299351 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Usage tracker for resource:security_group_rule and tenant:16a0d32c98e04307a5c57eacbbe14b54 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:51.307649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:51.309816 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=d974e0a5b2bf48cfad61f4928a055b27&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3798862 May 24 10:01:51.311035 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2759cd0b-0012-4f18-bfc7-2cd03d78b169,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:51.314286 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53668) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:51.335435 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.178s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:51.336710 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:01:51.366895 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:51.395312 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:51.422044 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c', 'code': 200} May 24 10:01:51.461779 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Allocated IP 172.27.0.19 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/97fff5ef-d87b-48e1-9d52-2ff0a4597cc4) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:01:51.462558 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.465241 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:51.476583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.514074 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.537171 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f6c95b0-8e0a-444f-819a-cfeee03322df req-7f2a1c53-18f2-48be-8a8e-9ec844e75eea tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:51.539234 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2f6c95b0-8e0a-444f-819a-cfeee03322df req-7f2a1c53-18f2-48be-8a8e-9ec844e75eea tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 944 time: 0.1882234 May 24 10:01:51.581515 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:51.589967 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:51.615094 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2f6c95b0-8e0a-444f-819a-cfeee03322df req-49d23ed2-5134-4fa1-879d-85b5704a1c15 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0670576 May 24 10:01:51.724703 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.730329 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:51.734962 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.745557 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2759cd0b-0012-4f18-bfc7-2cd03d78b169,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:51.749423 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:51.750388 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.751648 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/2759cd0b-0012-4f18-bfc7-2cd03d78b169 HTTP/1.1" status: 204 len: 168 time: 0.4361956 May 24 10:01:51.836719 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.868181 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.875011 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:51.934450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:51.949649 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b5e8118-778f-4670-8022-a573a706afc4 req-eac90d5f-e531-4eb0-9e5b-a881f97fd19c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:51.949649 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3b5e8118-778f-4670-8022-a573a706afc4 req-eac90d5f-e531-4eb0-9e5b-a881f97fd19c service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 1098 time: 0.0863404 May 24 10:01:51.980203 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:01:51.986353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:51.990827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Pushing event updated for resources: {'Port': ['ID=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:51.993356 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:01:52.007283 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3b5e8118-778f-4670-8022-a573a706afc4 req-59b980d1-ded0-4d5c-a4e5-41f38bfe17da service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0508950 May 24 10:01:52.045675 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.171s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:52.051531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.107s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:52.140016 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=16a0d32c98e04307a5c57eacbbe14b54&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9073572 May 24 10:01:52.142748 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53696) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:52.210094 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:52.242129 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:52.264308 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e59141b1-0163-4767-b3f7-ba325c9b256d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:52.269691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:52.399230 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:52.403879 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-aa0b004f-7a32-464d-afe1-0b65945e7d40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:52.417986 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e59141b1-0163-4767-b3f7-ba325c9b256d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:52.421657 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:52.426446 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:52.429715 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e59141b1-0163-4767-b3f7-ba325c9b256d HTTP/1.1" status: 204 len: 168 time: 0.2856715 May 24 10:01:52.605883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.046239 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.119140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c40081d9-10b8-4975-9121-0ca66bae26d3 req-8a0d7e9d-6e12-410f-8a55-8d1e9af18a3e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:53.120751 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c40081d9-10b8-4975-9121-0ca66bae26d3 req-8a0d7e9d-6e12-410f-8a55-8d1e9af18a3e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 945 time: 0.3197601 May 24 10:01:53.238244 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53736) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:53.258183 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.265340 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c40081d9-10b8-4975-9121-0ca66bae26d3 req-9a2606a5-72f7-4b47-84cf-4bcc03078a01 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1269803 May 24 10:01:53.277149 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:01:53.281461 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Usage tracker for resource:security_group_rule and tenant:e0df8551a78c41a0a5f842c3202bfd9e is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:01:53.288119 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:01:53.296579 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:01:53.392153 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.427618 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:01:53.444863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.453746 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:53.496029 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=e0df8551a78c41a0a5f842c3202bfd9e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2566979 May 24 10:01:53.498587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53746) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:53.510690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d71f9f07-c403-4af1-b5f8-75b6f06542e1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:53.514838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:53.569292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.597175 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.643140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:01:53.652179 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Sync routers for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:01:53.663679 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:53.663679 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:53.725104 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.726317 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 929 time: 3.6023993 May 24 10:01:53.734286 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53750) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:53.783892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.791507 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-190c18c3-7246-4315-a4b4-c7ecedcc3a4a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:53.830854 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d71f9f07-c403-4af1-b5f8-75b6f06542e1,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:53.830854 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:53.846347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:53.849467 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f604d431-2e1f-40f8-afd4-d4140bfab5ba admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/d71f9f07-c403-4af1-b5f8-75b6f06542e1 HTTP/1.1" status: 204 len: 168 time: 0.3474300 May 24 10:01:53.877216 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:54.372530 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:52Z', 'revision_number': 3, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:51Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:54.378817 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Checking router: 8eac1314-02e8-4975-a4e9-f33c258a1bd0 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:01:54.379394 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Updating router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:01:54.473387 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:54.611861 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53796) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:54.730140 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f13ea5d7-132d-49f0-9590-a23745072f34 req-0cd74baf-ed7f-4a96-befa-6b50e45395be tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:54.731449 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f13ea5d7-132d-49f0-9590-a23745072f34 req-0cd74baf-ed7f-4a96-befa-6b50e45395be tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 945 time: 0.2505958 May 24 10:01:54.749598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Deleting subnet bb5b8904-5b0e-454a-810f-cd13906648c5 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:01:54.782127 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:54.864991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f13ea5d7-132d-49f0-9590-a23745072f34 req-3f74be51-ef57-4268-b5eb-7c3f0217cb8c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1156957 May 24 10:01:54.876609 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:54.964716 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:54.990158 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.006316 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.021457 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Transition to ACTIVE for port object 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:01:55.146682 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.164523 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:55.234375 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Attempting to bind port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:01:55.235439 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Attempting to bind port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:01:55.236197 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Attempting to bind port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:01:55.256455 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:01:55.257224 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:01:55.257777 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.258279 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:01:55.259464 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Bound port: 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:01:55.259464 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.353144 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Pushing event updated for resources: {'Port': ['ID=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,revision_number=2']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:55.358236 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Allocated IP 172.20.0.1 (b0fff367-5ea3-4833-b1bf-44e4dd284af1/98c67f17-d401-4c79-8169-5da43506ba3c/b154e178-27fc-4709-b3e0-94ee2928fd20) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:01:55.366706 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.367134 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.367545 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.367944 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.368703 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.369107 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.370152 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.370811 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.371416 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.371976 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.372531 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.372965 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:01:55.376637 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] For port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:01:55.388064 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:55.389288 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] For port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:01:55.434738 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.439940 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-54f87b83-93b8-42da-8228-a50eeb1ec827 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:55.472054 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.477811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-331235b8-c4b8-49e7-b59f-c34b2f493023 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:55.592752 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=bb5b8904-5b0e-454a-810f-cd13906648c5,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:55.599552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:55.607591 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.611666 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.626056 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/bb5b8904-5b0e-454a-810f-cd13906648c5 HTTP/1.1" status: 204 len: 168 time: 1.0083480 May 24 10:01:55.639408 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53830) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:55.639716 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Pushing event updated for resources: {'Port': ['ID=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:55.639716 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-e2464c35-d3fb-4bd9-a223-d0fb732d6846 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.195s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:55.647582 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:55.648229 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:55.767101 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.833786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.841556 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:55.876885 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:55.916409 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.931360 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:01:55.934710 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:55.936447 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['aaad5285-e76e-499e-a2eb-1c9687521d4c']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:55.938461 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:01:55.938927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:55.957756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Pushing event updated for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:55.959502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '13331d89-de17-4f02-8e32-cd6b8ebcebba') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:55.959993 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '13331d89-de17-4f02-8e32-cd6b8ebcebba') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:55.961570 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:55.984604 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.002725 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.011529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Releasing vlan 92 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:01:56.012328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:01:56.032172 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:01:56.210783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.221227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.226714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.232913 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:56.241752 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:52Z', 'revision_number': 3, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:55Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:56.255255 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-77d85ed5-23a4-4222-9c79-7973e78c92b2 None None] The sync data for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi is: [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:52Z', 'revision_number': 3, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:55Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:01:56.261463 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] 10.222.0.67 "DELETE /v2.0/networks/aaad5285-e76e-499e-a2eb-1c9687521d4c HTTP/1.1" status: 204 len: 168 time: 0.6299527 May 24 10:01:56.262605 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Pushing event deleted for resources: {'Network': ['ID=aaad5285-e76e-499e-a2eb-1c9687521d4c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:56.265417 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:56.265975 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53844) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:56.357479 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Deleting subnet 45122598-0dab-4a6c-8a84-82955688c8b1 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:01:56.372175 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.445551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.503307 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.515888 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.765034 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.769645 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:56.770992 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f279a4f9-f946-4468-bc25-0b7a927d2bbc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:56.923235 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Pushing event updated for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:56.934022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:57.075475 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.076288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:01:57.078119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:01:57.089960 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:57.163426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=45122598-0dab-4a6c-8a84-82955688c8b1,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:57.186486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:57.191316 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.203866 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-844e912f-ad3e-433f-9411-289f4baf70d5 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/45122598-0dab-4a6c-8a84-82955688c8b1 HTTP/1.1" status: 204 len: 168 time: 0.9366090 May 24 10:01:57.205096 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53868) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:57.391346 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.434413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:01:57.569042 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.585074 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['da034945-4145-49ad-82fb-8d1318c66df8']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:57.598682 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '7d29ee2d-3e26-4b2e-8686-94d313ea291b') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:57.599121 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '7d29ee2d-3e26-4b2e-8686-94d313ea291b') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:57.622160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.647477 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.659091 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Releasing vlan 73 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:01:57.659759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:01:57.682580 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:01:57.690737 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:01:57.704506 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] 10.222.0.67 "PUT /v2.0/routers/8eac1314-02e8-4975-a4e9-f33c258a1bd0/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.9676843 May 24 10:01:57.907818 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.915784 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-6d9689f2-54be-4e68-8818-2fcd73cbbb2d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:57.943315 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:57.951328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Pushing event deleted for resources: {'Network': ['ID=da034945-4145-49ad-82fb-8d1318c66df8,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:57.960632 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] 10.222.0.67 "DELETE /v2.0/networks/da034945-4145-49ad-82fb-8d1318c66df8 HTTP/1.1" status: 204 len: 168 time: 0.7539680 May 24 10:01:57.961751 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:58.279653 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00ea914d-7b46-4567-b04a-0122bfb8dd17 req-a6ced3cd-8b38-4e6f-89d7-b8093c65f788 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:58.281461 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-00ea914d-7b46-4567-b04a-0122bfb8dd17 req-a6ced3cd-8b38-4e6f-89d7-b8093c65f788 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 945 time: 0.1089411 May 24 10:01:58.332648 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Sync routers for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:01:58.345639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:01:58.348297 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:58.401519 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-00ea914d-7b46-4567-b04a-0122bfb8dd17 req-0c1b3e2a-1380-4fc2-a012-b743a82ffc87 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1113346 May 24 10:01:58.443823 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:01:58.454688 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:01:58.460806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-8b3e9168-2b1a-4cd4-8759-0b9a20f782f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:58.462031 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-8b3e9168-2b1a-4cd4-8759-0b9a20f782f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:network {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:01:58.523851 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53910) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:58.587660 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f17d0206-b448-430f-8863-3bb4c88b75f5 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=bff351576a2f448e94b685ac4edad7c6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0621238 May 24 10:01:58.591057 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 53918) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:58.625651 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-8b3e9168-2b1a-4cd4-8759-0b9a20f782f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:network {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:01:58.627529 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-8b3e9168-2b1a-4cd4-8759-0b9a20f782f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 818 time: 0.4171793 May 24 10:01:58.661755 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:01:58.669993 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-41c16549-8195-4622-a57f-f89b9f33b5c8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/quotas/b08409afad374227ace1cdf31e4e9412 HTTP/1.1" status: 200 len: 375 time: 0.0306997 May 24 10:01:58.791837 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:58.851384 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-10713cba-3385-4a49-b328-6c09e97412be tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&fields=id HTTP/1.1" status: 200 len: 255 time: 0.1801972 May 24 10:01:58.877011 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:58.985079 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:01:58.985639 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-2f7164dc-541c-4e4c-aba6-f9f432c83d92 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/networks?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0416665 May 24 10:01:59.023748 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:01:59.024370 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.062203 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-010e5c3e-e367-43c0-a0da-6b00d9e602b0 service neutron] 10.222.0.67 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1594541 May 24 10:01:59.107742 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.120326 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7839c1a0-4080-470f-abfa-237d51c5edcb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:59.163975 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1c7ee109-b9fd-4acc-9511-fd28fd24333d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:59.205194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-b24eac08-cea1-4f78-84bb-563833673bd2 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:59.206665 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-b24eac08-cea1-4f78-84bb-563833673bd2 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.2202575 May 24 10:01:59.215702 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:56Z', 'revision_number': 4, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:55Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b154e178-27fc-4709-b3e0-94ee2928fd20', 'name': '', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'mac_address': 'fa:16:3e:74:21:5e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_interface', 'standard_attr_id': 300, 'fixed_ips': [{'subnet_id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:55Z', 'updated_at': '2021-05-24T10:01:56Z', 'revision_number': 2, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'subnets': [{'id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:01:59.218227 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.223549 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.227134 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/1c7ee109-b9fd-4acc-9511-fd28fd24333d HTTP/1.1" status: 204 len: 168 time: 0.6330996 May 24 10:01:59.231408 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:59.240627 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-99b98361-cf8d-4011-8b52-3a653312d84e tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/quotas/efa03a8a310f4bf9ba9e8166f64c9983 HTTP/1.1" status: 200 len: 375 time: 0.0222576 May 24 10:01:59.247586 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Checking router: 8eac1314-02e8-4975-a4e9-f33c258a1bd0 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:01:59.247960 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Updating router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 port b154e178-27fc-4709-b3e0-94ee2928fd20 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:01:59.275037 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-429a54ff-06ce-41a0-a999-2ce6394a1f65 service neutron] 10.222.0.67 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2051902 May 24 10:01:59.291114 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-391e54d0-51c5-4fb4-b682-be66f215e23b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0496185 May 24 10:01:59.319180 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:01:59.326756 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.443467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:01:59.450985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ee98fcc-2944-458d-87ea-bd996157ffdc None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:01:59.603274 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:01:59.609084 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:01:59.791780 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53960) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:01:59.868188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-0e5218fd-e9ec-4ca3-8315-12a66eb24c0f tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:01:59.883767 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-0e5218fd-e9ec-4ca3-8315-12a66eb24c0f tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 945 time: 0.2313657 May 24 10:01:59.971212 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e9b0f14-9302-42a2-9515-cb2a196b025b req-b50fd278-8975-4cd5-93e9-c06967f73423 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.1432931 May 24 10:02:00.062917 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.118525 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.134782 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Transition to ACTIVE for port object b154e178-27fc-4709-b3e0-94ee2928fd20 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:00.135660 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:00.140135 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Usage tracker for resource:security_group_rule and tenant:811245a3593d45aab0635a8ae8ac995f is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:00.144102 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-b5725364-f745-4564-b8a2-a309cd1f9c30 req-bc70dc7e-bbc6-43ca-a8d3-731863f4f4da tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:00.147706 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:00.156632 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:00.210035 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.230613 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:00.252360 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-b5725364-f745-4564-b8a2-a309cd1f9c30 req-bc70dc7e-bbc6-43ca-a8d3-731863f4f4da tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:00.253310 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b5725364-f745-4564-b8a2-a309cd1f9c30 req-bc70dc7e-bbc6-43ca-a8d3-731863f4f4da tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 210 time: 0.1668046 May 24 10:02:00.283884 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.340232 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f07d89f-a6c8-41d5-8f76-167a8247968b req-60365fa3-57b0-42f5-94ed-bde5b76e2c1f tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:00.348768 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6f07d89f-a6c8-41d5-8f76-167a8247968b req-60365fa3-57b0-42f5-94ed-bde5b76e2c1f tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 945 time: 0.3595302 May 24 10:02:00.365204 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53988) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:00.365990 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.382043 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Pushing event updated for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:00.384276 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ca52acf7-ce95-4955-b15d-9274843ebd09 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:00.418007 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Deleting port a98f21e3-a592-42a4-b016-d2cf72a32e5c {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:02:00.418374 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.481972 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6f07d89f-a6c8-41d5-8f76-167a8247968b req-f72520a5-c3ee-4eb6-846f-a3ff102e88f8 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1322906 May 24 10:02:00.504472 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=811245a3593d45aab0635a8ae8ac995f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7094069 May 24 10:02:00.516934 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 53992) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:00.620428 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.389s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:00.621945 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:00.624412 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=09dd7bc2-b771-4fdb-ba5d-3724077166ff,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:00.634332 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.250s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:00.690169 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-be89f3f8-39a5-4f14-9173-3cedb2cb0966 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c6ba64b66a6b46dfb69d5a4d7716d603&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3157351 May 24 10:02:00.692711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54008) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:00.700529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] For port a98f21e3-a592-42a4-b016-d2cf72a32e5c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=a98f21e3-a592-42a4-b016-d2cf72a32e5c,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:00.701180 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.895061 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Calling delete_port for a98f21e3-a592-42a4-b016-d2cf72a32e5c owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:02:00.903523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Attempting to bind port b154e178-27fc-4709-b3e0-94ee2928fd20 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:00.904804 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Attempting to bind port b154e178-27fc-4709-b3e0-94ee2928fd20 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:00.904804 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Attempting to bind port b154e178-27fc-4709-b3e0-94ee2928fd20 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:00.915570 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:00.915957 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Checking segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:00.916973 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.917906 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Bound using segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:00.918823 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Bound port: b154e178-27fc-4709-b3e0-94ee2928fd20, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:00.919214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:00.921400 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.022117 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.062401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:01.064827 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.066901 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.077915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-725bcd54-fbfd-4f64-a49e-d1c92b50bfd5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:01.087546 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-725bcd54-fbfd-4f64-a49e-d1c92b50bfd5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 818 time: 0.3931260 May 24 10:02:01.135210 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-0debff21-d3b5-44c8-8e08-4a6aa7d67168 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/quotas/b08409afad374227ace1cdf31e4e9412 HTTP/1.1" status: 200 len: 375 time: 0.0488565 May 24 10:02:01.180457 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.192787 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f272e6eb-94b0-4af8-8932-9f710656bc9e req-6eeb63ad-6c43-45cd-a668-177f44c408fa tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.0529418 May 24 10:02:01.199307 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.212976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:01.223976 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['a98f21e3-a592-42a4-b016-d2cf72a32e5c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:01.262476 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Pushing event deleted for resources: {'Port': ['ID=a98f21e3-a592-42a4-b016-d2cf72a32e5c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:01.272627 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-6e3b925c-2667-46af-8b3e-d208a93a1004 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&fields=id HTTP/1.1" status: 200 len: 255 time: 0.1339021 May 24 10:02:01.298790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] For port b154e178-27fc-4709-b3e0-94ee2928fd20, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:01.304709 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.312651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-410c7eea-bac5-49ea-a98a-6d4571261e38 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:01.343050 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=09dd7bc2-b771-4fdb-ba5d-3724077166ff,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:01.343947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:01.344326 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.344933 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] No DVR routers for this DVR port a98f21e3-a592-42a4-b016-d2cf72a32e5c on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:02:01.347882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.350785 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'name': 'network-vif-deleted', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:01.356877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:01.357500 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:01.390583 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] For port b154e178-27fc-4709-b3e0-94ee2928fd20, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b154e178-27fc-4709-b3e0-94ee2928fd20,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:01.408309 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '810ed8e9-80e4-4f0f-8d3c-9f64995bb42b', 'name': 'network-vif-deleted', 'tag': 'a98f21e3-a592-42a4-b016-d2cf72a32e5c', 'status': 'completed', 'code': 200} May 24 10:02:01.430399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.433563 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-43f49d2d-6ae7-490f-aa6a-fc8d46ffca35 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/09dd7bc2-b771-4fdb-ba5d-3724077166ff HTTP/1.1" status: 204 len: 168 time: 0.9098115 May 24 10:02:01.446841 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e929a24a-bbc1-4c2c-842b-db3ea90c3e4e req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "DELETE /v2.0/ports/a98f21e3-a592-42a4-b016-d2cf72a32e5c HTTP/1.1" status: 204 len: 173 time: 1.5555956 May 24 10:02:01.461371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.462471 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e5697d5f-1df8-4efb-b52b-67db58817699 HTTP/1.1" status: 204 len: 168 time: 0.7682385 May 24 10:02:01.463860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e5697d5f-1df8-4efb-b52b-67db58817699,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:01.468768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:01.483572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:01.492249 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:01.509756 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-901e62e9-08c1-4c86-b7aa-5859b17d924c service neutron] 10.222.0.67 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2324135 May 24 10:02:01.674955 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:02:01.675713 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:01:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:02:01.849814 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Pushing event updated for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:01.855959 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.363s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:01.907749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-228f7651-9400-4f9d-9183-d0cc5e11c1a5 req-0820093c-23a3-40e7-863d-a8a0ce0d8b12 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Synchronizing usage tracker for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:01.956063 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-34510981-32ea-4be7-8648-073be335081e service neutron] 10.222.0.67 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4454045 May 24 10:02:02.005168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-228f7651-9400-4f9d-9183-d0cc5e11c1a5 req-0820093c-23a3-40e7-863d-a8a0ce0d8b12 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Unset dirty status for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:02.006539 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-228f7651-9400-4f9d-9183-d0cc5e11c1a5 req-0820093c-23a3-40e7-863d-a8a0ce0d8b12 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.1455548 May 24 10:02:02.040220 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:02:02.396411 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-fe1836e6-8dca-44e2-a97c-65ebb8bdef8c req-03aa62ac-93c8-4fb3-a6da-bafe793566f2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.0546939 May 24 10:02:02.447734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:56Z', 'revision_number': 4, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b154e178-27fc-4709-b3e0-94ee2928fd20', 'name': '', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'mac_address': 'fa:16:3e:74:21:5e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_interface', 'standard_attr_id': 300, 'fixed_ips': [{'subnet_id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:55Z', 'updated_at': '2021-05-24T10:02:01Z', 'revision_number': 4, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'subnets': [{'id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:02:02.449094 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1dd9ea97-1d7a-4971-832c-a76d2760b25f req-9df83e64-b9a4-4ab9-823c-4ced00a4d990 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.0668235 May 24 10:02:02.452596 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c90056c6-6fc9-4fad-b29c-adc0e6022e88 None None] The sync data for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi is: [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:01:56Z', 'revision_number': 4, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b154e178-27fc-4709-b3e0-94ee2928fd20', 'name': '', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'mac_address': 'fa:16:3e:74:21:5e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_interface', 'standard_attr_id': 300, 'fixed_ips': [{'subnet_id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:55Z', 'updated_at': '2021-05-24T10:02:01Z', 'revision_number': 4, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'subnets': [{'id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:02:02.600163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54062) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:02.631042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:02.660103 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:02.665527 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Usage tracker for resource:security_group_rule and tenant:ab776d61a5684e1bb7dfc9b51dd1e6b7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:02.674231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:02.682288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:02.841120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54072) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:02.916805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:02.994142 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.008324 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c26744c7-c15f-43df-8f52-ba28e2b2e568 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:03.053748 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fa8bcc9d-5b9b-47d9-856f-c9ca91c6417e,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:03.057586 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:03.058319 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.065179 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ab776d61a5684e1bb7dfc9b51dd1e6b7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4626017 May 24 10:02:03.086443 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54086) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:03.136297 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:03.148976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Usage tracker for resource:security_group_rule and tenant:cd2ae423aea2459da3c87031b1a3bc05 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:03.162804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:03.177786 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:03.187814 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.266557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.306696 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4ecc7611-6c1e-47da-b883-22bea0d11c52 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.440586 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.444597 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f1cefd8c-3b0e-496b-a614-11552bb512a7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:03.470029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fa8bcc9d-5b9b-47d9-856f-c9ca91c6417e,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:03.476669 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:03.481495 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.481731 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-233f339f-6ab6-467b-a369-136d352acd2f req-6856696a-c0ee-4ecf-a74d-52b0b9c37979 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=810ed8e9-80e4-4f0f-8d3c-9f64995bb42b HTTP/1.1" status: 200 len: 210 time: 0.0857878 May 24 10:02:03.488812 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/fa8bcc9d-5b9b-47d9-856f-c9ca91c6417e HTTP/1.1" status: 204 len: 168 time: 0.3896499 May 24 10:02:03.582954 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-2b3d58ff-94a0-4d69-bf08-03dd42e4840f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:03.584556 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-2b3d58ff-94a0-4d69-bf08-03dd42e4840f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 818 time: 0.5970213 May 24 10:02:03.601243 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Request body: {'port': {'device_id': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'admin_state_up': True, 'tenant_id': 'b08409afad374227ace1cdf31e4e9412'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:03.782051 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3c951ef6-7bb4-48aa-8647-6d23653a7c27 req-dbf2bcfc-5f53-4c57-8aec-2c36d3dfeed1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.0629871 May 24 10:02:03.874692 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-426b245b-81c9-440a-ad2c-b08411186523 req-a1f1c073-fd3d-4453-ad7a-4137055f6f14 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.0938489 May 24 10:02:03.930357 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:03.950443 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:03.975156 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:04.022355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.044426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5034e2ea-ef5f-4095-926c-b2d41f2b2d5e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:04.108654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f77bfb1d-6fc4-4f0d-b526-e4381d8933f4,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:04.111900 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:04.118263 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=cd2ae423aea2459da3c87031b1a3bc05&name=default HTTP/1.1" status: 200 len: 2979 time: 1.2758892 May 24 10:02:04.120100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Made reservation on behalf of b08409afad374227ace1cdf31e4e9412 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:04.124606 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.126388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54122) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:04.256497 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.289300 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.358726 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-381cf3ea-736b-4c36-94bb-0511dcaf408b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.95 "GET /v2.0/networks?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0327439 May 24 10:02:04.426785 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b154e178-27fc-4709-b3e0-94ee2928fd20 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:04.474776 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Allocated IP 172.20.0.5 (b0fff367-5ea3-4833-b1bf-44e4dd284af1/98c67f17-d401-4c79-8169-5da43506ba3c/de553ab6-e238-403d-827d-b143492c1232) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:02:04.595521 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54132) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:04.617081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.642932 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:04.655423 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Usage tracker for resource:security_group_rule and tenant:e3f27ec08ed049eaa020772ef63b56e8 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:04.658490 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:04.661794 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-0a3f1941-5598-43f8-b7aa-1944d7f9be7b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:04.666137 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-0a3f1941-5598-43f8-b7aa-1944d7f9be7b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.3001692 May 24 10:02:04.666772 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:04.678502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Request body: {'port': {'device_id': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b', 'admin_state_up': True, 'tenant_id': 'efa03a8a310f4bf9ba9e8166f64c9983'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:04.684380 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port b154e178-27fc-4709-b3e0-94ee2928fd20 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:04.690402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port b154e178-27fc-4709-b3e0-94ee2928fd20 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:04.691651 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.744573 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.765288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3d573c6-81dd-4bf4-adc7-ee9dcd4990ce admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:04.788351 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:04.788935 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/f77bfb1d-6fc4-4f0d-b526-e4381d8933f4 HTTP/1.1" status: 204 len: 168 time: 0.6589715 May 24 10:02:04.796235 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f77bfb1d-6fc4-4f0d-b526-e4381d8933f4,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:04.931029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.166s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:04.936366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.016223 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device a98f21e3-a592-42a4-b016-d2cf72a32e5c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:02:05.069168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: a98f21e3-a592-42a4-b016-d2cf72a32e5c {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:02:05.069576 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device a98f21e3-a592-42a4-b016-d2cf72a32e5c not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:02:05.076901 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port b154e178-27fc-4709-b3e0-94ee2928fd20, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b154e178-27fc-4709-b3e0-94ee2928fd20,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:05.133309 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-95eb75e7-3763-49f1-be79-32d499b8e8ae req-601b33c7-4f45-4933-9d31-7020d7e98ebf tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.0669882 May 24 10:02:05.159728 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:05.165148 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Usage tracker for resource:port and tenant:efa03a8a310f4bf9ba9e8166f64c9983 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:05.168725 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Quota usage for port was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:05.201583 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.204398 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.208300 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:05.216504 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:05.252055 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.328523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:05.334414 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:05.369949 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d5bce59f-285f-4ecf-bf9f-64b46772cae2 req-15af572e-9754-4814-8ee1-714d650b3c3c tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.0850675 May 24 10:02:05.369949 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.382705 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-139ce189-a1c4-419b-829e-5720f54d15ad admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:05.395350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.482330 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-198eb089-02ff-4620-b729-8f35403722dd tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:05.484153 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-198eb089-02ff-4620-b729-8f35403722dd tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 818 time: 0.3092191 May 24 10:02:05.491904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-c403264e-380b-4f6e-9e7b-82bf36e6c07c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:05.493589 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=e3f27ec08ed049eaa020772ef63b56e8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.8948221 May 24 10:02:05.494847 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54166) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:05.504878 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b6157fe8-2d57-42a4-9081-457ddbf2c3f5,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:05.509390 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-c403264e-380b-4f6e-9e7b-82bf36e6c07c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 808 time: 0.5805812 May 24 10:02:05.514626 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Made reservation on behalf of efa03a8a310f4bf9ba9e8166f64c9983 for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:05.518000 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.528732 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:02:05.540570 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:05.541782 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.111s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:05.594841 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Request body: {'port': {'device_id': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'admin_state_up': True, 'tenant_id': 'b08409afad374227ace1cdf31e4e9412'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:05.607987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:05.614749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:02:05.651986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.686319 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:05.694648 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Pushing event updated for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:05.700865 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Usage tracker for resource:security_group_rule and tenant:efa03a8a310f4bf9ba9e8166f64c9983 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:05.704952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.164s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:05.709341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:05.717173 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:05.721123 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:05.721740 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.775050 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.796006 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-20588b4f-ee53-42ef-bef6-d3d1d10baaf1 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/quotas/6f38795dc42f4d5186770d3b222547ae HTTP/1.1" status: 200 len: 375 time: 0.2855735 May 24 10:02:05.804287 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.866683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.923328 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.3261316 May 24 10:02:05.929546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5a17f53a-2dc5-4cf5-a75b-f1b6cb982cbb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.941978 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Request body: {'port': {'device_id': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:05.942890 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-cbb14389-9ac5-4b5b-8378-6fcacd8883d5 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0905294 May 24 10:02:05.988057 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:05.996775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.014011 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3e459906-fca1-47aa-9cd1-a1890ead81d8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:06.031291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b6157fe8-2d57-42a4-9081-457ddbf2c3f5,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:06.037841 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:06.043135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.043135 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b6157fe8-2d57-42a4-9081-457ddbf2c3f5 HTTP/1.1" status: 204 len: 168 time: 0.5413947 May 24 10:02:06.082852 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54184) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:06.089932 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:06.095229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.109955 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5d069f79-7a46-4ba0-bff8-f3fb7270239a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:06.115133 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.119875 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Usage tracker for resource:port and tenant:b08409afad374227ace1cdf31e4e9412 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:06.137677 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:06.140019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.153717 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Usage tracker for resource:security_group_rule and tenant:d11063c7cbb143069a8746edc03cb93e is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:06.159022 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:06.166774 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Quota usage for port was recalculated. Used quota:2. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:06.185749 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:06.196945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:06.330040 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Pushing event updated for resources: {'SecurityGroup': ['ID=1e5cc604-d960-445e-ad7a-c0244613b030,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:06.338885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.230s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:06.343932 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-bafc0ea3-05b6-4dbc-aabb-f6d36668115c service neutron] 10.222.0.67 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3865614 May 24 10:02:06.354942 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b154e178-27fc-4709-b3e0-94ee2928fd20 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:06.363266 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a3bf4166-5514-4e58-b51a-65f640b11ba4 req-1d3b21ea-18fd-4b46-9bb5-17b7411e00af tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:06.491757 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Made reservation on behalf of b08409afad374227ace1cdf31e4e9412 for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:06.494685 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port b154e178-27fc-4709-b3e0-94ee2928fd20 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:06.495211 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.497080 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.521353 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a3bf4166-5514-4e58-b51a-65f640b11ba4 req-1d3b21ea-18fd-4b46-9bb5-17b7411e00af tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:06.531817 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3bf4166-5514-4e58-b51a-65f640b11ba4 req-1d3b21ea-18fd-4b46-9bb5-17b7411e00af tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.2217307 May 24 10:02:06.591811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.659062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Allocated IP 192.168.233.13 (2354deba-3cdc-41e6-b0d7-618ece22224b/4454f008-b32d-425f-94d3-001f5811f698/c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:02:06.676344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.724875 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8369ce50-394e-4d1c-8a1f-673df6674e8c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:06.827230 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-ab68a4ba-8d2b-40f8-978d-d0ea2121991a service neutron] 10.222.0.67 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4432418 May 24 10:02:06.836052 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=d11063c7cbb143069a8746edc03cb93e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7528219 May 24 10:02:06.849137 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54208) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:06.866549 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a5031560-ce16-4f77-9c43-81486f25c852 req-b9d442fc-6702-4fb1-86f2-e38fa3fa6d39 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.0770049 May 24 10:02:06.871686 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id de553ab6-e238-403d-827d-b143492c1232 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:06.875957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=89c52019-e251-4468-8634-926277a3e4f5,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:06.884750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:06.884750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:06.900340 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Transition to ACTIVE for port object de553ab6-e238-403d-827d-b143492c1232 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:07.062169 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.080771 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2e535d0e-f476-4c2d-838a-6a5c835026ab tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:07.221011 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.230551 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.233411 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.243846 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Allocated IP 172.20.0.6 (b0fff367-5ea3-4833-b1bf-44e4dd284af1/98c67f17-d401-4c79-8169-5da43506ba3c/c7f85b81-de77-4ee2-8223-a1fb751791f5) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:02:07.258128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Pushing event updated for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:07.277274 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.196s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:07.280574 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.288437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Transition to ACTIVE for port object c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:07.396698 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0d58bd49-54d8-4047-a5e7-d39b20ae79a6 req-30d972c7-8c68-47e6-b583-dd49c58418b3 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 210 time: 0.1536062 May 24 10:02:07.536797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.546355 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:07.561595 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=89c52019-e251-4468-8634-926277a3e4f5,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:07.595443 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:07.624040 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.631845 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Attempting to bind port de553ab6-e238-403d-827d-b143492c1232 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:07.632188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Attempting to bind port de553ab6-e238-403d-827d-b143492c1232 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:07.632550 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Attempting to bind port de553ab6-e238-403d-827d-b143492c1232 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:07.639880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Port de553ab6-e238-403d-827d-b143492c1232 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:02:07.639880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Attempting to bind port de553ab6-e238-403d-827d-b143492c1232 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:07.644993 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:07.646301 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:07.646690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Checking segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:07.647078 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Bound using segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:07.647529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Bound port: de553ab6-e238-403d-827d-b143492c1232, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:07.647908 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.763362 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:07.776679 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=de553ab6-e238-403d-827d-b143492c1232,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:07.792315 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Network 2354deba-3cdc-41e6-b0d7-618ece22224b is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:02:07.803410 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:07.806355 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:02:07.812371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:07.812998 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.825547 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-69863cc0-4b95-44fc-b49e-5f4bdfb884e4 req-5cd1c68e-927c-4a61-845d-91006b25f2c3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:07.834022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.848849 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Pushing event updated for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:07.852063 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.207s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:07.944830 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:07.944830 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/89c52019-e251-4468-8634-926277a3e4f5 HTTP/1.1" status: 204 len: 168 time: 1.0144873 May 24 10:02:08.146788 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.159173 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:08.176703 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-69863cc0-4b95-44fc-b49e-5f4bdfb884e4 req-5cd1c68e-927c-4a61-845d-91006b25f2c3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:08.186433 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-69863cc0-4b95-44fc-b49e-5f4bdfb884e4 req-5cd1c68e-927c-4a61-845d-91006b25f2c3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.4666631 May 24 10:02:08.187234 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.189853 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 937 time: 3.5174036 May 24 10:02:08.202289 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4fd35022-c6e0-4248-a34c-d8851949d5e1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.007s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:08.253015 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.253483 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.254829 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] 10.222.0.95 "PUT /v2.0/ports/de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 1093 time: 2.3203094 May 24 10:02:08.255464 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'tag': 'de553ab6-e238-403d-827d-b143492c1232'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:08.287832 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Pushing event updated for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:08.289191 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:08.297160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41998a04-14d7-4b49-93fc-427c8ac1785c req-d8b19bcd-bc6a-4b6a-884b-fdbde2cc69b1 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.299203 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-41998a04-14d7-4b49-93fc-427c8ac1785c req-d8b19bcd-bc6a-4b6a-884b-fdbde2cc69b1 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 935 time: 0.2834752 May 24 10:02:08.308817 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6ef61c83-a17f-42d2-b1e3-659d43a6a0d2 None None] Provisioning for port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:08.315641 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6ef61c83-a17f-42d2-b1e3-659d43a6a0d2 None None] Provisioning complete for port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:08.316324 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6ef61c83-a17f-42d2-b1e3-659d43a6a0d2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.349245 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'tag': 'de553ab6-e238-403d-827d-b143492c1232', 'status': 'completed', 'code': 200} May 24 10:02:08.370148 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-41998a04-14d7-4b49-93fc-427c8ac1785c req-d9fac34d-27d9-4881-8330-2f08ecd31ecb tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0576110 May 24 10:02:08.370412 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Pushing event updated for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:08.380822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Request body: {'port': {'device_id': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:08.421453 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-6ef61c83-a17f-42d2-b1e3-659d43a6a0d2 None None] Port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:02:08.452718 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.247s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:08.464546 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-a2a7644c-c195-4b44-89d7-1b790c1f9fde service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.467894 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-a2a7644c-c195-4b44-89d7-1b790c1f9fde service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1096 time: 0.1032026 May 24 10:02:08.512628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Network b0fff367-5ea3-4833-b1bf-44e4dd284af1 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:02:08.520259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:08.536122 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:02:08.542166 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.542467 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.558035 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c4366602-4071-4e8f-b94d-93d874d50805 req-1b53ab8a-a5da-4bfd-88d1-c1148902d00e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 210 time: 0.0880423 May 24 10:02:08.657278 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.682373 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 3.1680503 May 24 10:02:08.690487 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-c86c8714-330c-4c80-ae4c-80848dbe181a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.692671 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-c86c8714-330c-4c80-ae4c-80848dbe181a service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=b08409afad374227ace1cdf31e4e9412&shared=False HTTP/1.1" status: 200 len: 918 time: 0.2171557 May 24 10:02:08.721372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Request body: {'port': {'device_id': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:08.776829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:08.959624 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-ba78e9c5-7e1a-412a-87bd-6a893146056b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:08.959624 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f64a22be-f7d9-4064-a881-91387cd8e9d8 req-ba78e9c5-7e1a-412a-87bd-6a893146056b service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.2579002 May 24 10:02:09.010427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:09.177456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-020e548e-6fe5-4aa8-a920-f9432fcd852f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:09.178953 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-020e548e-6fe5-4aa8-a920-f9432fcd852f service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1096 time: 0.1832006 May 24 10:02:09.319627 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-b4c26a1b-50c6-43cd-be82-1b3fc15cb609 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 217 time: 0.1322999 May 24 10:02:09.625082 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-e52a11b8-ee49-41bb-95d2-af362293b449 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:09.626731 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-e52a11b8-ee49-41bb-95d2-af362293b449 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.3006482 May 24 10:02:09.689867 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-65887a82-7b1c-4a29-9849-f88069f959e6 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0550549 May 24 10:02:09.759413 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46550c11-9fd1-4e3b-b893-700104837e8f req-7c9f174c-b09c-4c6d-98ac-93511d775a6b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:09.759413 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-46550c11-9fd1-4e3b-b893-700104837e8f req-7c9f174c-b09c-4c6d-98ac-93511d775a6b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 931 time: 0.2941289 May 24 10:02:09.809574 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:09.863279 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:09.907331 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-46550c11-9fd1-4e3b-b893-700104837e8f req-bf02f14e-3e94-4898-9cff-0e3397dfbb48 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:09.916336 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-a803a499-794c-46b9-9f29-0a2d88b137e5 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2191246 May 24 10:02:09.921550 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Transition to ACTIVE for port object c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:09.944903 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id c7f85b81-de77-4ee2-8223-a1fb751791f5 {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:09.965335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:09.985555 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-077f2bb3-6ecf-4042-b43e-fef51c09398b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:09.986285 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-077f2bb3-6ecf-4042-b43e-fef51c09398b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Synchronizing usage tracker for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:10.039591 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.053184 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9a816cb4-3670-4392-8144-d00992cbb084 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:10.104167 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-290ba0c7-eb79-4843-b8eb-9a672626f70e req-f2a3ba51-0a48-4af6-9d7e-afa0cb548cfd service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1806393 May 24 10:02:10.107228 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-46550c11-9fd1-4e3b-b893-700104837e8f req-bf02f14e-3e94-4898-9cff-0e3397dfbb48 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:10.130356 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-46550c11-9fd1-4e3b-b893-700104837e8f req-bf02f14e-3e94-4898-9cff-0e3397dfbb48 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3052318 May 24 10:02:10.147004 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Transition to ACTIVE for port object c7f85b81-de77-4ee2-8223-a1fb751791f5 will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:10.155549 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-077f2bb3-6ecf-4042-b43e-fef51c09398b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Unset dirty status for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:10.164266 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-077f2bb3-6ecf-4042-b43e-fef51c09398b tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 935 time: 0.5374908 May 24 10:02:10.185285 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Pushing event updated for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:10.192148 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:10.301916 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.312290 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9095d849-97b3-494c-983e-f4732da90af7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:10.400232 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c421c1c3-c6ff-4f37-9d84-b1bd829701e9 req-dfbae925-3b40-43ab-99e4-db8f05549073 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 210 time: 0.0726864 May 24 10:02:10.411518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-b8382fc9-4ac6-4be2-bbcb-81e1c4f3f87f tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Synchronizing usage tracker for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:10.558104 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Pushing event updated for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:10.565198 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Attempting to bind port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:10.565658 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Attempting to bind port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:10.566072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Attempting to bind port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde on network 2354deba-3cdc-41e6-b0d7-618ece22224b {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:10.568749 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.257s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:10.608366 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde on network 2354deba-3cdc-41e6-b0d7-618ece22224b not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:02:10.608366 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Attempting to bind port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde on network 2354deba-3cdc-41e6-b0d7-618ece22224b {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:10.636256 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:10.636667 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Checking segment: {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:10.637162 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Bound using segment: {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:10.637693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Bound port: c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '34dfaf71-8e80-44a9-8573-f269e60764b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 54, 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:10.638112 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.666500 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Attempting to bind port c7f85b81-de77-4ee2-8223-a1fb751791f5 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:10.666962 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Attempting to bind port c7f85b81-de77-4ee2-8223-a1fb751791f5 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:10.667342 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Attempting to bind port c7f85b81-de77-4ee2-8223-a1fb751791f5 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:10.692875 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54346) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:10.728238 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-b8382fc9-4ac6-4be2-bbcb-81e1c4f3f87f tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Unset dirty status for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:10.729737 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-818cf9fa-754e-46a8-826e-d53d3be6f2d2 req-b8382fc9-4ac6-4be2-bbcb-81e1c4f3f87f tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.5645413 May 24 10:02:10.732340 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Port c7f85b81-de77-4ee2-8223-a1fb751791f5 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:02:10.732867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Attempting to bind port c7f85b81-de77-4ee2-8223-a1fb751791f5 on network b0fff367-5ea3-4833-b1bf-44e4dd284af1 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:10.742177 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:10.742715 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Checking segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:10.743183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Bound using segment: {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:10.743702 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Bound port: c7f85b81-de77-4ee2-8223-a1fb751791f5, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 71, 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:10.744677 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.803152 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-c0103ad7-b914-4958-94e9-6f632036b96e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:10.804591 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-c0103ad7-b914-4958-94e9-6f632036b96e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 808 time: 0.2506137 May 24 10:02:10.819336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Request body: {'port': {'device_id': '5b09368d-88d6-4178-8690-92edbf156805', 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15', 'admin_state_up': True, 'tenant_id': '6f38795dc42f4d5186770d3b222547ae'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:10.832055 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:10.845384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:10.870781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:10.876606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:10.952156 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.994315 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:10.998866 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:11.001954 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0167b504-3222-46f6-bf90-6ee9abe7289a tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:11.018718 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Usage tracker for resource:security_group_rule and tenant:6ba1817af8ce497ba1ca838a13fa2bb4 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:11.053311 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.097176 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.097942 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.099858 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] 10.222.0.95 "PUT /v2.0/ports/c7f85b81-de77-4ee2-8223-a1fb751791f5 HTTP/1.1" status: 200 len: 1093 time: 2.3837082 May 24 10:02:11.106684 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:11.118959 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.125288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.125288 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] 10.222.0.95 "PUT /v2.0/ports/c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde HTTP/1.1" status: 200 len: 1097 time: 2.7554004 May 24 10:02:11.147801 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Pushing event updated for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:11.160170 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:11.161110 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.096s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:11.161314 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:11.174569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:11.190682 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:11.213474 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:11.310890 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Pushing event updated for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:11.319640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:11.358115 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.376537 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Made reservation on behalf of 6f38795dc42f4d5186770d3b222547ae for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:11.380745 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.437716 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-9347184f-bbb3-4183-816a-9cf1c442ec92 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.439414 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-9347184f-bbb3-4183-816a-9cf1c442ec92 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 1096 time: 0.1871562 May 24 10:02:11.448630 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-6c6c6c87-d3c1-41ef-a451-5a5cccc48119 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.450121 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-6c6c6c87-d3c1-41ef-a451-5a5cccc48119 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 1100 time: 0.1055045 May 24 10:02:11.478796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.490989 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7bfb3dca-4e8b-4d7a-b016-db7a071106a8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:11.610501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6ba1817af8ce497ba1ca838a13fa2bb4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9101303 May 24 10:02:11.610501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54388) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:11.611050 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-d1d40071-77b4-4310-b8fc-3ee6d65beec7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5 HTTP/1.1" status: 200 len: 217 time: 0.1597989 May 24 10:02:11.611197 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=773a4997-4be4-42ed-b8d6-51e62f901e7d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:11.612561 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-1cb17eb0-bb84-4f9e-9daa-abe6315a82a0 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.13&port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde HTTP/1.1" status: 200 len: 217 time: 0.0913286 May 24 10:02:11.617817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:11.660200 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Allocated IP 172.20.0.4 (570c27de-a372-4133-bc81-83e9d6aa9b15/1f8e051e-bd56-4915-8e94-622f6ff108d3/5479c5f8-df8c-4e32-9a54-3dff2b992272) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:02:11.727311 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.793712 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:11.828541 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-93c232a6-6ea6-452d-be47-168caeeec933 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.830114 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-93c232a6-6ea6-452d-be47-168caeeec933 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.2043233 May 24 10:02:11.831482 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-b169bf52-6e84-424f-809b-2e5bcf7b1674 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:11.832756 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-b169bf52-6e84-424f-809b-2e5bcf7b1674 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=4454f008-b32d-425f-94d3-001f5811f698 HTTP/1.1" status: 200 len: 843 time: 0.2102742 May 24 10:02:11.921849 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-42170a40-62bc-4553-b58b-9bdc2622a3f7 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0798285 May 24 10:02:12.000089 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.051930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.057222 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a03c956d-3882-494c-aa50-2f8b5e820eb2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:12.077242 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=773a4997-4be4-42ed-b8d6-51e62f901e7d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:12.081194 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:12.105768 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5', 'status': 'completed', 'code': 200} May 24 10:02:12.153044 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.164567 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-71a626e7-6f27-43db-96f8-494140fe88f5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:12.215619 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-c0581252-866c-462b-9e9e-eefb4044f89b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:12.217163 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-c0581252-866c-462b-9e9e-eefb4044f89b service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=2354deba-3cdc-41e6-b0d7-618ece22224b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.3741469 May 24 10:02:12.226911 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-d611d530-2d66-4b2d-9790-931639f8ce60 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2883041 May 24 10:02:12.339947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Network 570c27de-a372-4133-bc81-83e9d6aa9b15 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:02:12.346271 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:12.349719 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:02:12.421262 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Pushing event updated for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:12.429543 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.268s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:12.432607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.434215 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/773a4997-4be4-42ed-b8d6-51e62f901e7d HTTP/1.1" status: 204 len: 168 time: 0.8198676 May 24 10:02:12.442775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79d24ccd-1057-4695-afc8-ba2f91413405 req-c3d89d28-9791-494b-938e-6bf9df27b9a1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:12.447571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:12.448202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.449810 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79d24ccd-1057-4695-afc8-ba2f91413405 req-c3d89d28-9791-494b-938e-6bf9df27b9a1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 943 time: 0.3376396 May 24 10:02:12.479834 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-707aab69-b636-4b0e-b5ed-36946a82da51 req-7f2f7df2-9dd5-40a1-bb6c-27365d1e2a1e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:12.486581 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-707aab69-b636-4b0e-b5ed-36946a82da51 req-7f2f7df2-9dd5-40a1-bb6c-27365d1e2a1e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Synchronizing usage tracker for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:12.542996 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.7289655 May 24 10:02:12.556792 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-108d26bc-3a07-4c2c-98d1-924a5a61e390 service neutron] 10.222.0.95 "GET /v2.0/networks/2354deba-3cdc-41e6-b0d7-618ece22224b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3267679 May 24 10:02:12.557451 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Request body: {'port': {'device_id': '5b09368d-88d6-4178-8690-92edbf156805', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:12.568676 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79d24ccd-1057-4695-afc8-ba2f91413405 req-5ec01cb4-5252-4001-944d-b00ae36f77cf tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1178730 May 24 10:02:12.575591 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82a78efd-99b9-4b1d-98b9-b317f1e9644a req-3a7f2228-9650-4921-8b40-dd3877d2696e service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3237259 May 24 10:02:12.665371 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-707aab69-b636-4b0e-b5ed-36946a82da51 req-7f2f7df2-9dd5-40a1-bb6c-27365d1e2a1e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Unset dirty status for tenant:6f38795dc42f4d5186770d3b222547ae on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:12.665371 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-707aab69-b636-4b0e-b5ed-36946a82da51 req-7f2f7df2-9dd5-40a1-bb6c-27365d1e2a1e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 931 time: 0.4038997 May 24 10:02:12.759379 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:12.831387 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-707aab69-b636-4b0e-b5ed-36946a82da51 req-6641a9a1-5dd3-4ff6-8a54-1b61027df4a1 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1372867 May 24 10:02:12.923556 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95a572c0-f201-418e-9a08-a075bb5186b8 req-076feca2-c847-4488-ae12-dbb8b2f898e6 service neutron] 10.222.0.95 "GET /v2.0/networks/2354deba-3cdc-41e6-b0d7-618ece22224b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3508375 May 24 10:02:12.950994 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07c9b5a9-a3ec-4791-a95e-37eb19036e2a req-cad94c36-dba2-49b3-a563-2328a55b21f3 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:12.952910 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-07c9b5a9-a3ec-4791-a95e-37eb19036e2a req-cad94c36-dba2-49b3-a563-2328a55b21f3 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 947 time: 0.2414212 May 24 10:02:13.001139 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-f1cd92be-bb50-447f-84ba-8e87e0a1c867 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:13.002561 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-f1cd92be-bb50-447f-84ba-8e87e0a1c867 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 1096 time: 0.2052410 May 24 10:02:13.005920 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device de553ab6-e238-403d-827d-b143492c1232 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:02:13.066383 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-07c9b5a9-a3ec-4791-a95e-37eb19036e2a req-c79a49f0-31a5-4cef-9b19-a8450df18523 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0979841 May 24 10:02:13.189774 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-579cdf95-30ef-43dd-9818-5620d435a697 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:13.190412 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=de553ab6-e238-403d-827d-b143492c1232,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:13.191813 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-579cdf95-30ef-43dd-9818-5620d435a697 service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.1544495 May 24 10:02:13.204653 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'de553ab6-e238-403d-827d-b143492c1232', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'port_id': 'de553ab6-e238-403d-827d-b143492c1232', 'mac_address': 'fa:16:3e:b5:07:47', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 71, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:02:13.211355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a968fe-fb25-4652-90c5-e2a647d5bd87 req-3cdaebca-6ffc-4490-8228-fa1bb5f5e696 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:13.222014 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a9a968fe-fb25-4652-90c5-e2a647d5bd87 req-3cdaebca-6ffc-4490-8228-fa1bb5f5e696 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2534373 May 24 10:02:13.273701 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.308890 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a9a968fe-fb25-4652-90c5-e2a647d5bd87 req-11b995bd-9190-4bb4-8c8c-c93145b9acd9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0856757 May 24 10:02:13.316338 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-e91dedf0-1185-461d-82a0-5b6a646268fb service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5 HTTP/1.1" status: 200 len: 217 time: 0.1068873 May 24 10:02:13.333350 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id de553ab6-e238-403d-827d-b143492c1232 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:13.371284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5479c5f8-df8c-4e32-9a54-3dff2b992272 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:13.375685 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=de553ab6-e238-403d-827d-b143492c1232,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:13.379423 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.384781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54422) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:13.398039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Transition to ACTIVE for port object 5479c5f8-df8c-4e32-9a54-3dff2b992272 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:13.405866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-27535d9d-ecec-4919-bc97-fee08a358f57 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:13.407251 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-27535d9d-ecec-4919-bc97-fee08a358f57 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.0829287 May 24 10:02:13.408123 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.444321 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.446090 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:13.462393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.464245 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-2f1b3596-0701-4fc5-9622-fe722c5cbf56 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0446777 May 24 10:02:13.465212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:13.473343 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Usage tracker for resource:security_group_rule and tenant:ae2d252e24d049eb88acd6303e2c87e3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:13.474691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:13.480463 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a58a7723-1271-4295-a77f-06776bdd2504 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:13.507025 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:13.548278 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:13.549607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Pushing event updated for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:13.563238 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:13.599511 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:13.657553 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-c47e270d-81ce-4d84-805f-46f25ab3fa57 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1842806 May 24 10:02:13.731763 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['de553ab6-e238-403d-827d-b143492c1232'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:13.763934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Attempting to bind port 5479c5f8-df8c-4e32-9a54-3dff2b992272 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:13.764306 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Attempting to bind port 5479c5f8-df8c-4e32-9a54-3dff2b992272 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:13.764623 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Attempting to bind port 5479c5f8-df8c-4e32-9a54-3dff2b992272 on network 570c27de-a372-4133-bc81-83e9d6aa9b15 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:13.770222 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Port 5479c5f8-df8c-4e32-9a54-3dff2b992272 on network 570c27de-a372-4133-bc81-83e9d6aa9b15 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:02:13.770486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Attempting to bind port 5479c5f8-df8c-4e32-9a54-3dff2b992272 on network 570c27de-a372-4133-bc81-83e9d6aa9b15 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:13.778371 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 1, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:13.778702 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Checking segment: {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:13.779080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Bound using segment: {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:13.780010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Bound port: 5479c5f8-df8c-4e32-9a54-3dff2b992272, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0af48d33-b0f2-492b-b676-993923c0f33a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:13.780339 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.983226 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:13.994739 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:14.000696 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:14.033582 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35930e75-feb1-45f4-a18d-b4f09e50ad7f req-89aa8ef6-8560-4fa4-975b-49a393c7af30 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3659523 May 24 10:02:14.047217 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41c96f08-27c1-4612-9aab-584c08435d30 req-4f032910-92c8-4e2e-ba2f-a1ebaf44bfa4 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.056183 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-41c96f08-27c1-4612-9aab-584c08435d30 req-4f032910-92c8-4e2e-ba2f-a1ebaf44bfa4 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 943 time: 0.2732229 May 24 10:02:14.079583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.079955 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.085146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-103f976c-6e95-409c-900a-ba2fcc199e1b tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:14.089884 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ae4a03ba-a300-4e5f-beb4-e9fc3864968b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:14.109122 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:14.144335 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-41c96f08-27c1-4612-9aab-584c08435d30 req-7e8fa8f0-8943-4ae7-ad6f-b495ee61fce1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0871592 May 24 10:02:14.151353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=71583828-00f6-4807-8f57-cc5b83da4bbd,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:14.154689 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:14.160568 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ae2d252e24d049eb88acd6303e2c87e3&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7746022 May 24 10:02:14.161124 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54442) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:14.206196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.207441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.208851 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] 10.222.0.95 "PUT /v2.0/ports/5479c5f8-df8c-4e32-9a54-3dff2b992272 HTTP/1.1" status: 200 len: 1093 time: 1.6569448 May 24 10:02:14.213820 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:14.232227 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.238456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Pushing event updated for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:14.242310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.157s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:14.261610 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f00eba5-7be4-4f29-b9b5-fe49e06ff8f5 req-3ffdebe5-322a-43f1-a653-20c590fd477e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.263493 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f00eba5-7be4-4f29-b9b5-fe49e06ff8f5 req-3ffdebe5-322a-43f1-a653-20c590fd477e tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 943 time: 0.1482501 May 24 10:02:14.265558 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.311084 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f00eba5-7be4-4f29-b9b5-fe49e06ff8f5 req-2bbff94e-0063-45a2-8642-b9ffd80eb8d4 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418856 May 24 10:02:14.383076 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde', 'status': 'completed', 'code': 200} May 24 10:02:14.391939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e0dbf61-8188-45fc-82d3-e6a24ed47b87 req-0d7d3ee2-48a1-4998-9670-c606335d4f08 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.393494 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3e0dbf61-8188-45fc-82d3-e6a24ed47b87 req-0d7d3ee2-48a1-4998-9670-c606335d4f08 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 947 time: 0.1321561 May 24 10:02:14.403280 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-6d8a1138-cb0a-49b3-a4bc-eadd76b29444 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.404834 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-6d8a1138-cb0a-49b3-a4bc-eadd76b29444 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 1096 time: 0.0918336 May 24 10:02:14.501468 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272', 'status': 'completed', 'code': 200} May 24 10:02:14.511972 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3e0dbf61-8188-45fc-82d3-e6a24ed47b87 req-942703d7-23f2-44a8-b1e7-5bf2c47189ff tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1100738 May 24 10:02:14.518011 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-9ac6a15a-6580-43ec-ba21-1c4ca1b05716 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272 HTTP/1.1" status: 200 len: 217 time: 0.0919051 May 24 10:02:14.518489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-885df5c3-95e9-480b-93aa-e67448560d07 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.519967 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-885df5c3-95e9-480b-93aa-e67448560d07 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 1100 time: 0.1009548 May 24 10:02:14.570503 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.577274 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d1b8c146-6eba-40be-9172-83951cc5029b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:14.592164 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=71583828-00f6-4807-8f57-cc5b83da4bbd,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:14.643340 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:14.645499 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:14.646626 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/71583828-00f6-4807-8f57-cc5b83da4bbd HTTP/1.1" status: 204 len: 168 time: 0.4843175 May 24 10:02:14.655656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-b978777c-6f68-4290-bdef-a95dc6dd5796 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.657265 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-b978777c-6f68-4290-bdef-a95dc6dd5796 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=1f8e051e-bd56-4915-8e94-622f6ff108d3 HTTP/1.1" status: 200 len: 848 time: 0.1382854 May 24 10:02:14.696987 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-73466c1c-c68f-403f-a4aa-ee68c8d59d37 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.699365 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-73466c1c-c68f-403f-a4aa-ee68c8d59d37 service neutron] 10.222.0.95 "GET /v2.0/networks?id=2354deba-3cdc-41e6-b0d7-618ece22224b HTTP/1.1" status: 200 len: 869 time: 0.1725745 May 24 10:02:14.709622 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-d76bf3f5-18a5-4ba4-bb2b-e1165e864ec0 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=570c27de-a372-4133-bc81-83e9d6aa9b15&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0444820 May 24 10:02:14.766144 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ef82e21-af24-4bf4-815c-f236cb4692c5 req-09888653-87b9-43ac-ad17-365985c3dce2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.767220 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ef82e21-af24-4bf4-815c-f236cb4692c5 req-09888653-87b9-43ac-ad17-365985c3dce2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2036705 May 24 10:02:14.797210 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-8089c375-8cf7-4fd5-9766-c196ed99640a service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.13&port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde HTTP/1.1" status: 200 len: 217 time: 0.0903215 May 24 10:02:14.871259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-954caf47-aad3-4717-b7cc-96693bb2733d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.872981 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-954caf47-aad3-4717-b7cc-96693bb2733d service neutron] 10.222.0.95 "GET /v2.0/subnets?id=4454f008-b32d-425f-94d3-001f5811f698 HTTP/1.1" status: 200 len: 843 time: 0.0690746 May 24 10:02:14.893637 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ef82e21-af24-4bf4-815c-f236cb4692c5 req-4a3db631-5a74-4b8d-ba6b-0d1934fc4c9e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1179447 May 24 10:02:14.904131 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-dd5aeb3c-4af6-4b32-be8c-169ca9bbdbdf service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1851497 May 24 10:02:14.964154 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-0ba77f16-7596-4e19-893d-cce93f68c117 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:14.965637 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-0ba77f16-7596-4e19-893d-cce93f68c117 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=2354deba-3cdc-41e6-b0d7-618ece22224b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.0858643 May 24 10:02:15.006354 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device de553ab6-e238-403d-827d-b143492c1232 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:15.044904 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-735733a6-cfbc-4744-9279-c8f98d8dec7c req-43035193-70f3-4880-a54f-94397bd8c7b1 service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1338830 May 24 10:02:15.155576 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-200fe277-a111-4e4c-bdb1-d77f51b013f4 service neutron] 10.222.0.95 "GET /v2.0/networks/2354deba-3cdc-41e6-b0d7-618ece22224b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1784508 May 24 10:02:15.222594 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port de553ab6-e238-403d-827d-b143492c1232 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:15.228942 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port de553ab6-e238-403d-827d-b143492c1232 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:15.229468 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:15.365915 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e3fc421f-7151-4734-9a63-3bc969f675bb req-45546525-5002-406f-a748-3a7bb08c364e service neutron] 10.222.0.95 "GET /v2.0/networks/2354deba-3cdc-41e6-b0d7-618ece22224b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1941521 May 24 10:02:15.366501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-4cd7ed15-8223-42af-86df-d6f9b1e21cc8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.368367 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-4cd7ed15-8223-42af-86df-d6f9b1e21cc8 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=6f38795dc42f4d5186770d3b222547ae&device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 1096 time: 0.1720881 May 24 10:02:15.439656 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:15.476133 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'de553ab6-e238-403d-827d-b143492c1232'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:15.484245 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=de553ab6-e238-403d-827d-b143492c1232,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:15.492372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6dbeaa61-c3d7-4015-aecb-75ea9e99b467 req-ffdb0170-eb1f-485b-98f0-d31ce590fc93 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.494049 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6dbeaa61-c3d7-4015-aecb-75ea9e99b467 req-ffdb0170-eb1f-485b-98f0-d31ce590fc93 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 943 time: 0.0969088 May 24 10:02:15.543317 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:15.545566 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:15.554836 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6dbeaa61-c3d7-4015-aecb-75ea9e99b467 req-555fc4a5-343c-4638-8c1f-e53728b01c28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0535243 May 24 10:02:15.616324 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'de553ab6-e238-403d-827d-b143492c1232', 'code': 200} May 24 10:02:15.619783 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-84f81297-c2b0-4034-a778-5a20b38b6cd3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.622094 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-84f81297-c2b0-4034-a778-5a20b38b6cd3 service neutron] 10.222.0.95 "GET /v2.0/networks?id=570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 200 len: 908 time: 0.2433646 May 24 10:02:15.654953 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:15.665230 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:15.709677 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-66abb4fd-9be8-4afe-bd25-09956c4eafba service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272 HTTP/1.1" status: 200 len: 217 time: 0.0768821 May 24 10:02:15.722437 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db217eec-47b7-4432-b4e1-643fe5cb05cd req-0e96ce1b-809c-473e-a80b-70b86a070c93 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.723877 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-db217eec-47b7-4432-b4e1-643fe5cb05cd req-0e96ce1b-809c-473e-a80b-70b86a070c93 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 943 time: 0.2198796 May 24 10:02:15.831715 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-db217eec-47b7-4432-b4e1-643fe5cb05cd req-e66465f8-ca80-4f42-9617-0a21991e21f0 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0774322 May 24 10:02:15.842046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f1ba865-409c-45e6-96e7-e1aec1cd0427 req-410408da-32fa-480d-b51b-d219ed56141e tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.845505 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6f1ba865-409c-45e6-96e7-e1aec1cd0427 req-410408da-32fa-480d-b51b-d219ed56141e tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 947 time: 0.1132567 May 24 10:02:15.849429 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-001d1f56-0033-42a9-8e1b-66c48ad4b68c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:15.850853 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-001d1f56-0033-42a9-8e1b-66c48ad4b68c service neutron] 10.222.0.95 "GET /v2.0/subnets?id=1f8e051e-bd56-4915-8e94-622f6ff108d3 HTTP/1.1" status: 200 len: 848 time: 0.1261294 May 24 10:02:15.915318 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6f1ba865-409c-45e6-96e7-e1aec1cd0427 req-b125bf30-8948-47d1-8038-be469565d369 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0656438 May 24 10:02:15.920126 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-40957b49-68f1-4089-ab83-7332d295a658 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=570c27de-a372-4133-bc81-83e9d6aa9b15&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0611501 May 24 10:02:16.075990 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0906d9e3-1479-4e32-92b2-e979e9ff8509 req-79892372-2a9d-447e-b1c8-3c715f5fe7f3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:16.077689 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0906d9e3-1479-4e32-92b2-e979e9ff8509 req-79892372-2a9d-447e-b1c8-3c715f5fe7f3 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.0985680 May 24 10:02:16.122692 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-0906d9e3-1479-4e32-92b2-e979e9ff8509 req-85e6ba2b-0e83-4aa5-83b3-0fcaeb320fce service neutron] Synchronizing usage tracker for tenant:ea4946fb0c044fb8ac328ee9fc158b53 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:16.154662 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-4bad7e72-e01e-44a8-9165-24795bdee1d6 service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2272327 May 24 10:02:16.227735 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-0906d9e3-1479-4e32-92b2-e979e9ff8509 req-85e6ba2b-0e83-4aa5-83b3-0fcaeb320fce service neutron] Unset dirty status for tenant:ea4946fb0c044fb8ac328ee9fc158b53 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:16.229028 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0906d9e3-1479-4e32-92b2-e979e9ff8509 req-85e6ba2b-0e83-4aa5-83b3-0fcaeb320fce service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1448584 May 24 10:02:16.370542 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5abbf9e-b5ca-4f01-97bd-d6b8d207dc37 req-86f3154b-5436-4c4d-8e83-11332776baa6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:16.373621 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b5abbf9e-b5ca-4f01-97bd-d6b8d207dc37 req-86f3154b-5436-4c4d-8e83-11332776baa6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.3260713 May 24 10:02:16.454360 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b5abbf9e-b5ca-4f01-97bd-d6b8d207dc37 req-74967295-e837-4008-be04-5c76da81849d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0748525 May 24 10:02:16.457901 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a3f3588b-a9cf-4306-9539-95d2d341fd46 req-823ba60a-790f-48f4-b666-42ffdbdcd711 service neutron] 10.222.0.95 "GET /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2947574 May 24 10:02:16.747359 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-741e4012-a06f-4b3e-8cfa-82db32648396 req-7d377570-99ca-40a0-a012-8362fb3c6f4c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:16.748883 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-741e4012-a06f-4b3e-8cfa-82db32648396 req-7d377570-99ca-40a0-a012-8362fb3c6f4c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 943 time: 0.0861154 May 24 10:02:16.796079 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-741e4012-a06f-4b3e-8cfa-82db32648396 req-259e6394-e57f-4948-b89f-2dbf30c50bd3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414042 May 24 10:02:17.020027 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:02:17.125083 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68243c13-4def-4814-80a3-4bc98ab1f086 req-9e809fbd-b949-4c06-8260-172fdf1c45c0 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:17.126631 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68243c13-4def-4814-80a3-4bc98ab1f086 req-9e809fbd-b949-4c06-8260-172fdf1c45c0 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 943 time: 0.1883354 May 24 10:02:17.181160 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fc178e7-806a-4101-9566-96dd2f711db9 req-6701c913-5547-4d3b-8f68-8e2f7b4ea185 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:17.183354 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7fc178e7-806a-4101-9566-96dd2f711db9 req-6701c913-5547-4d3b-8f68-8e2f7b4ea185 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 947 time: 0.1627274 May 24 10:02:17.204256 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68243c13-4def-4814-80a3-4bc98ab1f086 req-416a89a4-f320-4fb5-b26a-315b6abdb4cf tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0700312 May 24 10:02:17.241634 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7fc178e7-806a-4101-9566-96dd2f711db9 req-7dceafcb-b55e-43b0-b020-d7964213c278 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0514843 May 24 10:02:17.305911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:17.320448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde', 'network_id': '2354deba-3cdc-41e6-b0d7-618ece22224b', 'port_id': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde', 'mac_address': 'fa:16:3e:03:d7:51', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 54, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4454f008-b32d-425f-94d3-001f5811f698', 'ip_address': '192.168.233.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:02:17.382091 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:17.399327 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:17.418196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:17.472905 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:17.473770 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c7f85b81-de77-4ee2-8223-a1fb751791f5 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:02:17.477252 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:17.576987 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:17.581497 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:17.685684 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e7e2090-cdbf-4840-a4ae-22a9b4c3c348 req-ea093a3f-d929-4319-b22a-be3a05eba030 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:17.687313 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e7e2090-cdbf-4840-a4ae-22a9b4c3c348 req-ea093a3f-d929-4319-b22a-be3a05eba030 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0963244 May 24 10:02:17.688304 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:17.702881 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'c7f85b81-de77-4ee2-8223-a1fb751791f5', 'network_id': 'b0fff367-5ea3-4833-b1bf-44e4dd284af1', 'port_id': 'c7f85b81-de77-4ee2-8223-a1fb751791f5', 'mac_address': 'fa:16:3e:30:85:ca', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 71, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '98c67f17-d401-4c79-8169-5da43506ba3c', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:02:17.740442 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2e7e2090-cdbf-4840-a4ae-22a9b4c3c348 req-055c377e-a542-424b-b3db-3415ea7df905 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0450380 May 24 10:02:17.767959 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:17.784712 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id c7f85b81-de77-4ee2-8223-a1fb751791f5 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:17.801924 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:17.895910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:17.897895 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:17.969176 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:17.972264 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:17.991605 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6badd4c3-0286-46dd-814a-df5451a38779 req-f5142776-94e8-430b-8159-036ff47a19bd tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:17.993213 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6badd4c3-0286-46dd-814a-df5451a38779 req-f5142776-94e8-430b-8159-036ff47a19bd tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.0912590 May 24 10:02:18.058111 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6badd4c3-0286-46dd-814a-df5451a38779 req-fa8c9f47-afe4-4697-800d-45058ab735bf tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0585799 May 24 10:02:18.145242 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:18.193005 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['c7f85b81-de77-4ee2-8223-a1fb751791f5'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:18.437182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-961dc026-d839-4c45-b45e-6ca9487e36fc req-ad47a654-51e7-469b-8040-69d8ede13222 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:18.439487 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-961dc026-d839-4c45-b45e-6ca9487e36fc req-ad47a654-51e7-469b-8040-69d8ede13222 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 943 time: 0.1043360 May 24 10:02:18.530495 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84115b32-af3b-437f-b745-97bf7fe8eff9 req-97dafce0-34aa-48d6-9caf-dab3b0349906 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:18.532009 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-84115b32-af3b-437f-b745-97bf7fe8eff9 req-97dafce0-34aa-48d6-9caf-dab3b0349906 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 948 time: 0.1245773 May 24 10:02:18.569238 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-961dc026-d839-4c45-b45e-6ca9487e36fc req-9563f1a6-e2a3-4d33-98fb-fa8023ad045a tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1223915 May 24 10:02:18.621797 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-84115b32-af3b-437f-b745-97bf7fe8eff9 req-9638d348-49c6-4585-bf7b-8677a4fa9fac tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0838087 May 24 10:02:18.947906 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-754bc133-fe32-4374-a825-69bfbf616686 req-8b58e67b-9179-486e-ba59-e6cb393fc4b1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:18.949382 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-754bc133-fe32-4374-a825-69bfbf616686 req-8b58e67b-9179-486e-ba59-e6cb393fc4b1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0872774 May 24 10:02:18.997355 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-754bc133-fe32-4374-a825-69bfbf616686 req-23ea9e0b-9e12-42d1-be3b-42c00dafc650 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0417850 May 24 10:02:19.271274 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6544fae3-cf09-4408-a72e-2f65471dbacc req-8a899fa8-8e74-477d-9409-1a2536b07a7e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:19.272853 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6544fae3-cf09-4408-a72e-2f65471dbacc req-8a899fa8-8e74-477d-9409-1a2536b07a7e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.0918534 May 24 10:02:19.284574 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54580) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:19.323251 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6544fae3-cf09-4408-a72e-2f65471dbacc req-43ad15a8-caee-4072-960d-2ad1cbd65364 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0440807 May 24 10:02:19.407751 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.426583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:19.430437 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Usage tracker for resource:security_group_rule and tenant:67af59cfa2ef4cd8a9717557a9377ee7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:19.433797 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:19.439476 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:19.518208 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.656145 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.665120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-126eb5f7-4259-4dfb-9d8e-e744d7a7f066 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:19.718516 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 5479c5f8-df8c-4e32-9a54-3dff2b992272 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:02:19.761312 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=67af59cfa2ef4cd8a9717557a9377ee7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4755065 May 24 10:02:19.762154 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=40431f41-fa25-492d-bc44-ecb747c10b67,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:19.764479 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54594) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:19.769144 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:19.850999 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2621ac49-fce6-4622-96f6-0e8e22701805 req-8f5ab1b9-4dbb-45d0-9079-cf7665234d1a tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:19.852514 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2621ac49-fce6-4622-96f6-0e8e22701805 req-8f5ab1b9-4dbb-45d0-9079-cf7665234d1a tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 948 time: 0.1025591 May 24 10:02:19.859591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b53b49dd-b54e-4fd7-b478-fb44007d4a09 req-68b29f40-bdb4-40fc-8514-d0527a4c1663 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:19.861179 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b53b49dd-b54e-4fd7-b478-fb44007d4a09 req-68b29f40-bdb4-40fc-8514-d0527a4c1663 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 943 time: 0.1728182 May 24 10:02:19.865539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.894609 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:19.899259 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2621ac49-fce6-4622-96f6-0e8e22701805 req-0ee545f8-2534-42ba-8661-9a0c645e50ce tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410101 May 24 10:02:19.909203 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '5479c5f8-df8c-4e32-9a54-3dff2b992272', 'network_id': '570c27de-a372-4133-bc81-83e9d6aa9b15', 'port_id': '5479c5f8-df8c-4e32-9a54-3dff2b992272', 'mac_address': 'fa:16:3e:44:d0:13', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1f8e051e-bd56-4915-8e94-622f6ff108d3', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:02:19.920772 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.948845 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b53b49dd-b54e-4fd7-b478-fb44007d4a09 req-86b0a227-13d7-48fb-bb74-8a84151b2d1a tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0812881 May 24 10:02:19.964749 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:19.993566 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5479c5f8-df8c-4e32-9a54-3dff2b992272 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:20.010302 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:20.050272 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.054155 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-241a002f-432c-4fc5-904a-68402ee4bb05 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:20.065168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=40431f41-fa25-492d-bc44-ecb747c10b67,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:20.069318 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:20.069468 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.073189 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.075304 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-79917e72-e02c-453b-9150-8efd5f174fb6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:20.076984 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/40431f41-fa25-492d-bc44-ecb747c10b67 HTTP/1.1" status: 204 len: 168 time: 0.3093233 May 24 10:02:20.107208 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.156253 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:20.159946 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:20.198813 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0e06b98-183e-48b1-a2fe-d82e935c2c84 req-1290fbe2-d4a1-4154-ad53-cbeb252b4908 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:20.200300 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0e06b98-183e-48b1-a2fe-d82e935c2c84 req-1290fbe2-d4a1-4154-ad53-cbeb252b4908 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0892584 May 24 10:02:20.245245 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0e06b98-183e-48b1-a2fe-d82e935c2c84 req-7bce08cd-e81b-417b-8423-c8cf1249c835 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390255 May 24 10:02:20.508305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ae0fdc3-c1c9-4081-83df-6dc4b3c515fa req-32f075e8-05d0-479c-bd30-d4e0ccc94119 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:20.509823 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae0fdc3-c1c9-4081-83df-6dc4b3c515fa req-32f075e8-05d0-479c-bd30-d4e0ccc94119 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.0798416 May 24 10:02:20.556212 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae0fdc3-c1c9-4081-83df-6dc4b3c515fa req-082de336-04cc-401b-ae55-aa685a50ccea tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0407031 May 24 10:02:20.657390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54610) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:20.675109 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.689775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:20.693739 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Usage tracker for resource:security_group_rule and tenant:3f8a76b985154a33810a354cd8dd8cc5 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:20.696598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:20.702804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:20.794552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.861556 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:20.870608 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8dea803c-527b-4fac-9708-6f6c5243ae59 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:20.910298 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=3f8a76b985154a33810a354cd8dd8cc5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2513933 May 24 10:02:20.911869 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ccbffaf1-3fd7-40cd-bd8b-be7477eed957,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:20.912963 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54616) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:20.914999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:20.973698 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:21.000881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:21.118925 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad6a55de-4ed6-4125-b6d4-d6127ec1cf95 req-153b57ab-9a35-4686-90ed-cb13fa706a3c tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:21.121664 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ad6a55de-4ed6-4125-b6d4-d6127ec1cf95 req-153b57ab-9a35-4686-90ed-cb13fa706a3c tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 948 time: 0.1084557 May 24 10:02:21.138291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f22df52b-f546-45aa-81dc-8b5a603f15e0 req-085a8c03-8018-4eef-8634-340db3d6a1bb tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:21.139831 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f22df52b-f546-45aa-81dc-8b5a603f15e0 req-085a8c03-8018-4eef-8634-340db3d6a1bb tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 944 time: 0.0841026 May 24 10:02:21.169540 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ad6a55de-4ed6-4125-b6d4-d6127ec1cf95 req-b8e9a6c0-a292-4a7a-ae06-26930011f23d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416465 May 24 10:02:21.186447 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:21.187454 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:21.194718 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f22df52b-f546-45aa-81dc-8b5a603f15e0 req-d2f06e35-5fb8-4d60-aa8f-b2ed8b175155 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0482402 May 24 10:02:21.200790 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ccbffaf1-3fd7-40cd-bd8b-be7477eed957,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:21.203625 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:21.204551 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:21.209694 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/ccbffaf1-3fd7-40cd-bd8b-be7477eed957 HTTP/1.1" status: 204 len: 168 time: 0.2917092 May 24 10:02:21.439129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1fcaee01-1a5f-43e6-b753-55fab51905e0 req-4fae3928-a502-4c2d-80ab-6e2a492a80f4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:21.440677 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1fcaee01-1a5f-43e6-b753-55fab51905e0 req-4fae3928-a502-4c2d-80ab-6e2a492a80f4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0858641 May 24 10:02:21.494773 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1fcaee01-1a5f-43e6-b753-55fab51905e0 req-37aa2ccc-cf1f-4422-9e15-a3292210b3f9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0473828 May 24 10:02:21.744875 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-602000f8-75a0-420a-b900-0b56d2072957 req-0ca25ff4-eaf3-4975-84a7-4d6e464e8a63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:21.746429 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-602000f8-75a0-420a-b900-0b56d2072957 req-0ca25ff4-eaf3-4975-84a7-4d6e464e8a63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.0848703 May 24 10:02:21.796378 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-602000f8-75a0-420a-b900-0b56d2072957 req-de6b6bf9-f83e-4f48-8f87-61b787d42e4c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0442755 May 24 10:02:22.153238 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['5479c5f8-df8c-4e32-9a54-3dff2b992272'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:22.401895 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-913eeac5-53f9-4324-8198-ac486070de47 req-13452c92-30d7-4c9b-a78f-31b3a814fbd8 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:22.409604 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-913eeac5-53f9-4324-8198-ac486070de47 req-13452c92-30d7-4c9b-a78f-31b3a814fbd8 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 948 time: 0.1242878 May 24 10:02:22.461648 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-913eeac5-53f9-4324-8198-ac486070de47 req-bf5c9f4b-f990-45af-a83b-f909ae740553 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0507197 May 24 10:02:22.466964 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-164fe730-aafd-45b0-813b-5064ae3eadb4 req-38c98c8c-183a-4d12-b701-08b64584c84c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:22.468761 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-164fe730-aafd-45b0-813b-5064ae3eadb4 req-38c98c8c-183a-4d12-b701-08b64584c84c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 944 time: 0.0877469 May 24 10:02:22.521562 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-164fe730-aafd-45b0-813b-5064ae3eadb4 req-c75cb4a3-af9c-499f-8779-5eea9ff95541 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0460348 May 24 10:02:22.716164 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c98b024-3b0b-43b7-ad92-21cd6aa95590 req-54e69dc2-2fad-48f4-b6e8-e68f509277d6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:22.717655 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4c98b024-3b0b-43b7-ad92-21cd6aa95590 req-54e69dc2-2fad-48f4-b6e8-e68f509277d6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0889771 May 24 10:02:22.763328 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4c98b024-3b0b-43b7-ad92-21cd6aa95590 req-bfda55d4-22f5-4279-945b-dd8e3932a08f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0397301 May 24 10:02:23.017871 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b5f6ca4-fbaa-495f-bf13-aa0a0643d581 req-48d4c8a2-affc-4a01-b9c1-9344c59dd498 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:23.020836 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5b5f6ca4-fbaa-495f-bf13-aa0a0643d581 req-48d4c8a2-affc-4a01-b9c1-9344c59dd498 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.1066844 May 24 10:02:23.074153 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5b5f6ca4-fbaa-495f-bf13-aa0a0643d581 req-28bce191-e4be-4c67-a4c9-d6e9f32dc8e5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0489886 May 24 10:02:23.495526 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e22fccd-796f-4749-a4b4-3639ab2e943d req-c3d9a993-4dc7-406f-803a-96e26db044b1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:23.497134 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e22fccd-796f-4749-a4b4-3639ab2e943d req-c3d9a993-4dc7-406f-803a-96e26db044b1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0920296 May 24 10:02:23.564575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e22fccd-796f-4749-a4b4-3639ab2e943d req-f9c159ef-fbcb-4040-9fd9-3c3cbc76a395 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0543015 May 24 10:02:23.605879 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:23.668157 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-12a6214a-af1b-4591-8206-823ad5048fe6 req-88a33db4-d417-4576-9740-bf35eb3e3e76 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:23.669719 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-12a6214a-af1b-4591-8206-823ad5048fe6 req-88a33db4-d417-4576-9740-bf35eb3e3e76 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 948 time: 0.0913098 May 24 10:02:23.718811 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-12a6214a-af1b-4591-8206-823ad5048fe6 req-a4832201-1889-4006-b1a2-5bedcb5e33ee tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434682 May 24 10:02:23.750816 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f57955b-b9a0-4c0d-917d-3ec03df455ce req-c86883d0-b543-429a-93b6-ae0e81d0dd58 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:23.752275 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2f57955b-b9a0-4c0d-917d-3ec03df455ce req-c86883d0-b543-429a-93b6-ae0e81d0dd58 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 944 time: 0.0823772 May 24 10:02:23.796967 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2f57955b-b9a0-4c0d-917d-3ec03df455ce req-89ebc06d-a078-4ee8-852a-5a4afee1f4b6 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389423 May 24 10:02:23.947121 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:23.952215 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:23.952593 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:23.961784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8afef5a1-7d3d-4c93-a87a-3fc8adc436e6 req-3f958604-b78f-4f11-bbda-76358c13050d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:23.963753 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8afef5a1-7d3d-4c93-a87a-3fc8adc436e6 req-3f958604-b78f-4f11-bbda-76358c13050d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0799406 May 24 10:02:24.019170 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8afef5a1-7d3d-4c93-a87a-3fc8adc436e6 req-e11cb270-8efa-4247-8e07-c8c9b1cacc7a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0493183 May 24 10:02:24.097418 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.135391 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:24.145610 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:24.187311 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde', 'code': 200} May 24 10:02:24.193002 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.194947 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 5479c5f8-df8c-4e32-9a54-3dff2b992272 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:24.230220 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:24.337698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:24.341441 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:24.372587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f28fc0f-02f3-4772-a4df-6ce584ade695 req-078241cc-7e1b-4d65-a617-cb7e9083a519 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:24.382914 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 5479c5f8-df8c-4e32-9a54-3dff2b992272 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:24.383252 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6f28fc0f-02f3-4772-a4df-6ce584ade695 req-078241cc-7e1b-4d65-a617-cb7e9083a519 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 944 time: 0.1335943 May 24 10:02:24.389666 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 5479c5f8-df8c-4e32-9a54-3dff2b992272 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:24.390187 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.444247 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6f28fc0f-02f3-4772-a4df-6ce584ade695 req-0af8fe47-5dff-4de2-9d4c-365698a62131 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0541849 May 24 10:02:24.527195 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.566444 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:24.619570 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96480caf-46a0-4e27-91bb-308191e8f0cf req-c98d1078-aab0-450b-9eb7-c5516233bdfd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:24.621238 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-96480caf-46a0-4e27-91bb-308191e8f0cf req-c98d1078-aab0-450b-9eb7-c5516233bdfd service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 1102 time: 0.0767264 May 24 10:02:24.645891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.650483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c7f85b81-de77-4ee2-8223-a1fb751791f5 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:24.675630 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-96480caf-46a0-4e27-91bb-308191e8f0cf req-a77e144c-8d3c-44f2-a75f-9b97d44cd087 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0458393 May 24 10:02:24.689368 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:24.787227 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-415a97bb-0e2b-45d8-8a0d-59d86942be69 req-95d7bb33-4795-4842-8be2-38b0cc7ef6af tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:24.788735 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-415a97bb-0e2b-45d8-8a0d-59d86942be69 req-95d7bb33-4795-4842-8be2-38b0cc7ef6af tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1011732 May 24 10:02:24.794550 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:24.805677 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:24.832718 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port c7f85b81-de77-4ee2-8223-a1fb751791f5 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:02:24.839373 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-415a97bb-0e2b-45d8-8a0d-59d86942be69 req-6f8bb8ba-6ca8-4e9a-bfef-849353f3c120 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0424213 May 24 10:02:24.840053 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port c7f85b81-de77-4ee2-8223-a1fb751791f5 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:24.840431 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:24.972763 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:25.017411 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:25.045688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ce06315-81ac-4091-b7af-71358db7cac1 req-f3fb0711-29c4-4c30-a168-cff1e10d02af tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:25.047426 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6ce06315-81ac-4091-b7af-71358db7cac1 req-f3fb0711-29c4-4c30-a168-cff1e10d02af tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.0975313 May 24 10:02:25.062615 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f370bfa-67f3-4550-bbe9-c0c75979bcde req-f2dff050-97c9-477a-9df4-613afaba75eb tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:25.064121 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f370bfa-67f3-4550-bbe9-c0c75979bcde req-f2dff050-97c9-477a-9df4-613afaba75eb tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 949 time: 0.2104855 May 24 10:02:25.078976 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:25.081131 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5f7055d7-4d10-4971-87e7-1e2ab97782a6 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:25.116488 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f370bfa-67f3-4550-bbe9-c0c75979bcde req-1c5af941-9280-42e3-9748-9bc4164117da tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0457635 May 24 10:02:25.116778 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6ce06315-81ac-4091-b7af-71358db7cac1 req-c6df27c0-8982-4329-be69-b43240a9d6d7 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0623922 May 24 10:02:25.159063 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:25.162270 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:25.317256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-881c3ce2-97e3-4a66-8bed-91192fcbd441 req-76b6a8cd-a2a1-41e5-bd83-706313217af8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:25.322800 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-881c3ce2-97e3-4a66-8bed-91192fcbd441 req-76b6a8cd-a2a1-41e5-bd83-706313217af8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0910802 May 24 10:02:25.374450 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-881c3ce2-97e3-4a66-8bed-91192fcbd441 req-47c39ea2-d216-41cd-a955-274ab08103c7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411811 May 24 10:02:25.737820 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:02:25.738708 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71554155-6cec-439e-8c63-da738fa2cfd5 req-b845eb12-c0e4-415b-a06b-35b9d6e296be tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:25.738708 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-71554155-6cec-439e-8c63-da738fa2cfd5 req-b845eb12-c0e4-415b-a06b-35b9d6e296be tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0980215 May 24 10:02:25.738708 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54788) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:25.775532 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:25.793658 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-71554155-6cec-439e-8c63-da738fa2cfd5 req-0e70e524-f19d-4fc9-a422-0aa9f30440d7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0540652 May 24 10:02:25.797087 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:02:25.877204 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:25.930793 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:25.949537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:25.963770 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:25.979723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:25.984174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Usage tracker for resource:security_group_rule and tenant:b7056442820345cba551c17376698ce6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:25.987246 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:25.993345 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:26.089206 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.112799 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4c8689c-8fcd-42f3-8e48-1bacb2c00040 req-5977e3a6-2704-4ee2-b2ac-65aba4f983ec tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:26.114412 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f4c8689c-8fcd-42f3-8e48-1bacb2c00040 req-5977e3a6-2704-4ee2-b2ac-65aba4f983ec tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1081572 May 24 10:02:26.165415 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f4c8689c-8fcd-42f3-8e48-1bacb2c00040 req-a3bb00e9-29df-4f98-8863-c1b5d14aa105 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0430605 May 24 10:02:26.192353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.192474 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272'}, {'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:26.198514 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.200380 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:26.207848 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-35991423-c2b5-458e-9294-6190cce9738c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:26.260856 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b7056442820345cba551c17376698ce6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5252087 May 24 10:02:26.260856 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d5cb8d06-1280-4c49-8101-b6a61cffb542,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:26.261100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 54818) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:26.297347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.297588 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.297795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.298118 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.298470 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.298816 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.299160 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.299502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.299842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.300234 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.300576 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.313284 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:26.353035 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['6fec3696-3d1d-4762-9674-fe87fd37e211'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:26.385675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=6']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:26.396434 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.403298 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.203s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:26.439906 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.232s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:26.493564 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f6e35b6-538b-48db-824b-210a9679dc14 req-e7c3dab6-d83f-411b-b65a-018f8a708323 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:26.495381 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f6e35b6-538b-48db-824b-210a9679dc14 req-e7c3dab6-d83f-411b-b65a-018f8a708323 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 949 time: 0.1493359 May 24 10:02:26.498942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.606714 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f6e35b6-538b-48db-824b-210a9679dc14 req-19c78400-b325-4f05-b3e3-e0aab945a72f tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1028595 May 24 10:02:26.638025 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b796e369-572f-40dd-bc8b-0daf555fa42d req-6462749b-6602-4efb-b345-2411dc365af9 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:26.649591 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b796e369-572f-40dd-bc8b-0daf555fa42d req-6462749b-6602-4efb-b345-2411dc365af9 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.2336640 May 24 10:02:26.681354 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272', 'code': 200} May 24 10:02:26.681632 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5', 'code': 200} May 24 10:02:26.778755 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b796e369-572f-40dd-bc8b-0daf555fa42d req-6e60070b-6d29-4e77-a38c-9f2449f8de6f tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1270275 May 24 10:02:26.941407 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:26.945882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a0aeea6c-8939-46a0-bc0a-95468cee71da admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:26.974987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d5cb8d06-1280-4c49-8101-b6a61cffb542,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:27.007930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.008501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.008892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.009325 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.016756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:27.034756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:27.035805 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:27.039454 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/d5cb8d06-1280-4c49-8101-b6a61cffb542 HTTP/1.1" status: 204 len: 168 time: 0.7749581 May 24 10:02:27.134979 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a1d20cc-7d3a-4a01-96d6-b7038cf3b3a9 req-264c5cb9-916c-41b8-aa7d-9ef335067911 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:27.136525 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7a1d20cc-7d3a-4a01-96d6-b7038cf3b3a9 req-264c5cb9-916c-41b8-aa7d-9ef335067911 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 945 time: 0.2551594 May 24 10:02:27.235718 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7a1d20cc-7d3a-4a01-96d6-b7038cf3b3a9 req-8076c4a4-af2f-4f64-8a75-d13e5ed6247d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0847933 May 24 10:02:27.270406 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-994a782b-7291-4dca-81ce-914ae5f5227a req-65392d29-ef6e-413b-ba87-686e4872fb98 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:27.271825 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-994a782b-7291-4dca-81ce-914ae5f5227a req-65392d29-ef6e-413b-ba87-686e4872fb98 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 1098 time: 0.0994239 May 24 10:02:27.399119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c97c1c98-7c12-4b5b-95d1-abb88b4d85d4 req-5bd85d60-f33a-4ba7-b374-bdeb9de53e2f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:27.404721 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c97c1c98-7c12-4b5b-95d1-abb88b4d85d4 req-5bd85d60-f33a-4ba7-b374-bdeb9de53e2f service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 1098 time: 0.1878896 May 24 10:02:27.406498 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-994a782b-7291-4dca-81ce-914ae5f5227a req-0f7dce9b-93b7-4f91-9a6e-40012d112b68 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1271989 May 24 10:02:27.543404 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c97c1c98-7c12-4b5b-95d1-abb88b4d85d4 req-671412a5-2fae-4276-8f52-5c4ec279240d service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0655818 May 24 10:02:27.605852 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56b90044-3af7-45eb-8215-4c7f6089f4c8 req-cfeaccbc-9707-4d72-9e84-d1b9213de521 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:27.611556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:02:27.620554 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-56b90044-3af7-45eb-8215-4c7f6089f4c8 req-cfeaccbc-9707-4d72-9e84-d1b9213de521 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1913478 May 24 10:02:27.735354 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-56b90044-3af7-45eb-8215-4c7f6089f4c8 req-186e410f-8e7e-4674-87f9-cc31d9bbb4f9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1131830 May 24 10:02:27.838712 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 6fec3696-3d1d-4762-9674-fe87fd37e211 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:02:27.839496 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:28.054145 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-229c6c80-c3a6-4439-b233-926624cc69ec req-ecd19cbc-f7bf-470a-a996-80cf4b7289b1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:28.054145 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-229c6c80-c3a6-4439-b233-926624cc69ec req-ecd19cbc-f7bf-470a-a996-80cf4b7289b1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2343886 May 24 10:02:28.054145 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-229c6c80-c3a6-4439-b233-926624cc69ec req-f619dd48-c2f3-4991-be9a-1cdd8ad45b61 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0735233 May 24 10:02:28.062007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:28.110715 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-705c3ec3-e136-4172-905a-1680bb623fea req-050a9ffe-d481-4d6d-a9b5-700181015c3d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:28.122052 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-705c3ec3-e136-4172-905a-1680bb623fea req-050a9ffe-d481-4d6d-a9b5-700181015c3d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 949 time: 0.1676927 May 24 10:02:28.149105 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:28.227747 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:28.229865 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-9fc5dfa1-63b4-4fa8-9473-5d2f21901ac1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:28.246998 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-705c3ec3-e136-4172-905a-1680bb623fea req-5859de13-b866-4597-a549-887b5aad369d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1238213 May 24 10:02:28.352818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=7']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:28.357424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:28.395172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54864) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:28.432636 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:28.439950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cd27412-8f65-45ac-b914-d4108e7712e0 req-df1d3bef-b051-4992-8e68-33d9ba2d9825 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:28.444309 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1cd27412-8f65-45ac-b914-d4108e7712e0 req-df1d3bef-b051-4992-8e68-33d9ba2d9825 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.2481422 May 24 10:02:28.531402 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1cd27412-8f65-45ac-b914-d4108e7712e0 req-b10e6622-4088-44b7-abc7-d974c48ab7c3 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0705481 May 24 10:02:28.572098 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:28.579043 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Usage tracker for resource:security_group_rule and tenant:e4ecc66989db4196a7e2a25e49649d0b is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:28.583657 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:28.597368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:28.685529 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:28.718661 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-89917f97-2de3-424f-9483-0ae73b340296 req-a879dfb2-d02f-46e8-8dc3-7cf79ad93fab tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:28.728124 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-89917f97-2de3-424f-9483-0ae73b340296 req-a879dfb2-d02f-46e8-8dc3-7cf79ad93fab tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2771595 May 24 10:02:28.755626 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'code': 200} May 24 10:02:28.833161 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-89917f97-2de3-424f-9483-0ae73b340296 req-2660dcc5-48bd-4ea8-9b90-1271125cf7c5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1016381 May 24 10:02:28.887797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:28.956607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f864de77-0a17-4391-bd37-dce8cf379850 req-cbf8a0fe-fb56-4c6b-9b77-5976d72caa6e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:28.958939 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f864de77-0a17-4391-bd37-dce8cf379850 req-cbf8a0fe-fb56-4c6b-9b77-5976d72caa6e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0947397 May 24 10:02:29.016167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:29.027629 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1eb64f22-4762-4ea9-be58-9c7060fd682f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:29.040722 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f864de77-0a17-4391-bd37-dce8cf379850 req-5be64f89-a539-4f67-96e7-eb11c2e91f55 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0740852 May 24 10:02:29.129736 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=845bbd79-5e95-4610-bffd-96a00c1d3221,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:29.151726 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=e4ecc66989db4196a7e2a25e49649d0b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7604365 May 24 10:02:29.153835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:29.154313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 54898) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:29.291623 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:29.388793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:29.515648 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a27be366-b2ee-45ab-b970-15518c69ca3d req-fa76c436-8ad5-4540-9124-2fad1abc03ac tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:29.517482 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a27be366-b2ee-45ab-b970-15518c69ca3d req-fa76c436-8ad5-4540-9124-2fad1abc03ac tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2129006 May 24 10:02:29.533174 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c969c4f-08e0-4e19-9fc2-7db9d9917c14 req-2ef918f2-e0fa-400b-a30d-059880c4c452 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:29.544126 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c969c4f-08e0-4e19-9fc2-7db9d9917c14 req-2ef918f2-e0fa-400b-a30d-059880c4c452 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 949 time: 0.1135716 May 24 10:02:29.568688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:29.583309 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1048e8b9-34df-4ae3-9bd0-6e4012dda8ba admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:29.607058 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c969c4f-08e0-4e19-9fc2-7db9d9917c14 req-07b24000-18a1-4a10-9ff7-e8d08caf68ee tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/security-groups?id=1e5cc604-d960-445e-ad7a-c0244613b030&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0617733 May 24 10:02:29.607397 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=845bbd79-5e95-4610-bffd-96a00c1d3221,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:29.631179 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a27be366-b2ee-45ab-b970-15518c69ca3d req-b78a5370-8509-4ae9-9cc3-811ed42cfe2b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0974197 May 24 10:02:29.631179 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:29.658068 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-eba2e5ad-58b1-40bf-bf37-ba5e2efbe544 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:29.659857 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-eba2e5ad-58b1-40bf-bf37-ba5e2efbe544 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.95 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 949 time: 0.2328300 May 24 10:02:29.714462 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:29.715813 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/845bbd79-5e95-4610-bffd-96a00c1d3221 HTTP/1.1" status: 204 len: 168 time: 0.5605505 May 24 10:02:29.817858 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0f24c24-2a2f-41d0-a678-611b8bb2a6f5 req-327266a9-b249-43c8-9082-d80b58ef4583 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:29.826876 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a0f24c24-2a2f-41d0-a678-611b8bb2a6f5 req-327266a9-b249-43c8-9082-d80b58ef4583 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.1061833 May 24 10:02:29.896233 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a0f24c24-2a2f-41d0-a678-611b8bb2a6f5 req-9c7821cc-1975-4be4-89eb-9003b3695cf2 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0679965 May 24 10:02:29.936229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Deleting port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:02:29.936673 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:30.188045 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] For port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,segment=NetworkSegment(34dfaf71-8e80-44a9-8573-f269e60764b0),segment_id=34dfaf71-8e80-44a9-8573-f269e60764b0)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:30.188662 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:30.360670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1347ebb9-6255-4a32-a479-311ff182af66 req-376b586a-1eb1-4003-b316-5e7e75269c48 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:30.367741 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1347ebb9-6255-4a32-a479-311ff182af66 req-376b586a-1eb1-4003-b316-5e7e75269c48 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1276367 May 24 10:02:30.381669 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Calling delete_port for c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde owned by compute:nova {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:02:30.402717 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1be33a0-4bb5-4831-b65d-e2384f98fd76 req-9f9f8c49-0781-4fd0-b862-23f63f69050e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:30.413171 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1be33a0-4bb5-4831-b65d-e2384f98fd76 req-9f9f8c49-0781-4fd0-b862-23f63f69050e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.3485744 May 24 10:02:30.429575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1347ebb9-6255-4a32-a479-311ff182af66 req-49bbe8a5-647c-4d87-8dfa-da15330297fe tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0607634 May 24 10:02:30.505680 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1be33a0-4bb5-4831-b65d-e2384f98fd76 req-b6990563-b7f2-4a2e-aa74-8cb6085c82c1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0914948 May 24 10:02:30.759296 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:30.766215 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:30.771818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:30.835563 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Pushing event deleted for resources: {'Port': ['ID=c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:30.850860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:31.190202 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-04fc18b0-dfbc-46cd-877a-825026f73e10 req-5a7b39c7-7ed4-4d7f-b4cd-ed83a6a37f2a tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.3289509 May 24 10:02:31.194217 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] No DVR routers for this DVR port c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde on host hv-cinder-79123 {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:02:31.198483 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:31.202224 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'name': 'network-vif-deleted', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:31.323448 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78602f70-0293-43a4-91e3-4c2b7da68e0c req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.95 "DELETE /v2.0/ports/c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde HTTP/1.1" status: 204 len: 173 time: 1.6319551 May 24 10:02:31.615591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c45459c-d4d1-47ff-9adc-a291cd388ca3 req-161c45c9-74d9-44a4-b276-e03efba2d0ae tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:31.654707 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9c45459c-d4d1-47ff-9adc-a291cd388ca3 req-161c45c9-74d9-44a4-b276-e03efba2d0ae tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2002413 May 24 10:02:31.713755 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9c45459c-d4d1-47ff-9adc-a291cd388ca3 req-770ce570-aff3-4135-8b13-c80a62268eb8 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0702772 May 24 10:02:31.716799 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-09899195-c134-40ef-98ca-dd97ebee12a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:31.718312 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-09899195-c134-40ef-98ca-dd97ebee12a9 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 1098 time: 0.3765099 May 24 10:02:31.761225 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c81aefec-746a-4dcd-9d18-bd3fe4ab469f req-ae8085e6-1431-479b-847d-27b8c34facde tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:31.762728 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c81aefec-746a-4dcd-9d18-bd3fe4ab469f req-ae8085e6-1431-479b-847d-27b8c34facde tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1607249 May 24 10:02:31.803004 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c8ea541a-eca6-424a-be23-a7324dd192c7', 'name': 'network-vif-deleted', 'tag': 'c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde', 'status': 'completed', 'code': 200} May 24 10:02:31.929633 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c81aefec-746a-4dcd-9d18-bd3fe4ab469f req-840d5f34-cda6-47e7-af67-13ab606ac15b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1267548 May 24 10:02:31.929633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-5182dfe0-8b73-42a2-ad5c-943d78b4d6cc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:31.929633 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-5182dfe0-8b73-42a2-ad5c-943d78b4d6cc service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.1998360 May 24 10:02:31.987049 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b2727390-121e-420b-84a0-d67627c3f2c5 req-381a97b2-efa5-456b-b50f-5e569846f683 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:31.997766 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2727390-121e-420b-84a0-d67627c3f2c5 req-381a97b2-efa5-456b-b50f-5e569846f683 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0998223 May 24 10:02:32.017266 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-51bab1de-94a7-4e99-a2da-451d4a933951 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5 HTTP/1.1" status: 200 len: 217 time: 0.0762081 May 24 10:02:32.055837 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2727390-121e-420b-84a0-d67627c3f2c5 req-8def5555-63b0-4ff5-8271-6f046037b4fc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0610094 May 24 10:02:32.102274 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-43830952-5a74-4ae5-9724-64ab6db52079 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:32.103983 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-43830952-5a74-4ae5-9724-64ab6db52079 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.0804579 May 24 10:02:32.165089 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-ede0c68e-f9d6-4c4a-ba98-e6940827faff service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0445318 May 24 10:02:32.554279 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-ea8eb18d-2a42-4f55-b9cf-8d648a1d709e service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3853724 May 24 10:02:32.561928 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-396c442c-aed7-4d1c-bb80-c745a2f81702 req-a69c77b1-1416-4db1-8393-5859f3f8ae6d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Synchronizing usage tracker for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:32.764609 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-396c442c-aed7-4d1c-bb80-c745a2f81702 req-a69c77b1-1416-4db1-8393-5859f3f8ae6d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Unset dirty status for tenant:efa03a8a310f4bf9ba9e8166f64c9983 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:32.773224 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-396c442c-aed7-4d1c-bb80-c745a2f81702 req-a69c77b1-1416-4db1-8393-5859f3f8ae6d tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] 10.222.0.67 "GET /v2.0/ports?device_id=c8ea541a-eca6-424a-be23-a7324dd192c7 HTTP/1.1" status: 200 len: 210 time: 0.2875679 May 24 10:02:32.835687 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d118837-854b-4a58-9a1d-3dd0485d1122 req-d98c25a6-c9e0-4976-b2d8-a1a116f86a4f service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2756972 May 24 10:02:33.003770 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2a6d4612-155f-4947-b72a-b214cc4c1475 req-fd8705a6-55eb-445f-8e6c-f1ed136533e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:33.038118 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2a6d4612-155f-4947-b72a-b214cc4c1475 req-fd8705a6-55eb-445f-8e6c-f1ed136533e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1456175 May 24 10:02:33.121847 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2a6d4612-155f-4947-b72a-b214cc4c1475 req-dfeadfd3-f144-45e1-8111-2d6f5a2195b0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0826039 May 24 10:02:33.285886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ce7ac5e6-2e8f-4518-98d3-b75fb227ac03 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:33.376052 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4af74177-9488-4345-8650-c9f5a63faa2f req-f05b1403-9e2e-4836-9474-3c93bb05bd25 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:33.383524 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4af74177-9488-4345-8650-c9f5a63faa2f req-f05b1403-9e2e-4836-9474-3c93bb05bd25 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1218941 May 24 10:02:33.456984 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4af74177-9488-4345-8650-c9f5a63faa2f req-754df171-e736-4ee4-b8ea-2d75c8375b22 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0678413 May 24 10:02:33.529229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab38bf5f-5c1d-40b4-9e83-0fd889786ffc req-2ae00383-231b-4897-ac24-b07a7fb76840 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:33.548481 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab38bf5f-5c1d-40b4-9e83-0fd889786ffc req-2ae00383-231b-4897-ac24-b07a7fb76840 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1991696 May 24 10:02:33.633814 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:02:33.645090 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ab38bf5f-5c1d-40b4-9e83-0fd889786ffc req-6ec6d3e8-e6d7-43f4-a70f-3f4ae28800a8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0847213 May 24 10:02:33.718173 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:02:33.718531 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c643f4f5-6c4c-4b8b-8ba5-cbe94def0fde not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:02:33.766303 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b550e2a6-eb4c-470a-9343-1ec5455c7d2c req-6089395c-ed87-4358-b88d-de2b3fb33205 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:33.770339 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b550e2a6-eb4c-470a-9343-1ec5455c7d2c req-6089395c-ed87-4358-b88d-de2b3fb33205 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.0984221 May 24 10:02:33.780233 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 55044) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:33.851981 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b550e2a6-eb4c-470a-9343-1ec5455c7d2c req-10e20ff6-f48b-462f-8675-cdfdb5c2f590 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0692356 May 24 10:02:33.991866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.012834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:34.017262 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Usage tracker for resource:security_group_rule and tenant:5ee7ee1a059f4e4da1c66a421d15c95c is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:34.020444 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:34.026895 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:34.125241 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.198245 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.208573 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f55bdf7c-2b0e-4c0a-8bb0-7bc07830eb3a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:34.255977 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3d1831f9-3a99-4faa-adf0-71452b96caaf,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:34.265374 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:34.278367 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=5ee7ee1a059f4e4da1c66a421d15c95c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4971230 May 24 10:02:34.280958 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55060) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:34.424260 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.462241 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.479999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1daf1a57-f992-4e8c-bd2e-7a4476d93b87 req-8e6a7b4d-b4e7-417f-a54e-2ce31d5382f4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:34.481712 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1daf1a57-f992-4e8c-bd2e-7a4476d93b87 req-8e6a7b4d-b4e7-417f-a54e-2ce31d5382f4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1835206 May 24 10:02:34.581682 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1daf1a57-f992-4e8c-bd2e-7a4476d93b87 req-56ed77a7-6616-41b0-8115-7922fc81db5c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0864871 May 24 10:02:34.662921 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.675934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1887e0b1-a563-4da9-a536-7a3826ebf4f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:34.757606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9176ff0d-7488-4a8b-a8a9-e20a6ca87976 req-856a8aa4-fdac-4e61-84a1-cfee1af2da53 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:34.759578 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9176ff0d-7488-4a8b-a8a9-e20a6ca87976 req-856a8aa4-fdac-4e61-84a1-cfee1af2da53 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1433060 May 24 10:02:34.795065 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3d1831f9-3a99-4faa-adf0-71452b96caaf,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:34.802021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:34.803401 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/3d1831f9-3a99-4faa-adf0-71452b96caaf HTTP/1.1" status: 204 len: 168 time: 0.5209172 May 24 10:02:34.805300 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:34.882821 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9176ff0d-7488-4a8b-a8a9-e20a6ca87976 req-b4950795-f65e-4f13-ac57-49886511a50e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0586975 May 24 10:02:34.928837 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f7086f3-859e-4ca6-8a21-7f6af18b2c4d req-61f4c9c5-e248-4b2b-8a25-874d2c5fe3ec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:34.930608 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-f6148f67-e5ee-4bb2-94a2-82b940a05e54 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:34.937701 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-f6148f67-e5ee-4bb2-94a2-82b940a05e54 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.1719472 May 24 10:02:34.937808 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f7086f3-859e-4ca6-8a21-7f6af18b2c4d req-61f4c9c5-e248-4b2b-8a25-874d2c5fe3ec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1417360 May 24 10:02:35.026462 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f7086f3-859e-4ca6-8a21-7f6af18b2c4d req-d77ad4e6-8119-4746-8787-d21ededd667b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0863285 May 24 10:02:35.263190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bab34692-c8f9-4fed-9aef-6bd1a6960df8 req-03686096-738a-4ddc-8c0e-ecf1a62c6e9d tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:35.264782 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bab34692-c8f9-4fed-9aef-6bd1a6960df8 req-03686096-738a-4ddc-8c0e-ecf1a62c6e9d tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 945 time: 0.2052996 May 24 10:02:35.266627 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Deleting port 5479c5f8-df8c-4e32-9a54-3dff2b992272 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:02:35.267070 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:35.447614 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:02:35.508859 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bab34692-c8f9-4fed-9aef-6bd1a6960df8 req-46f25143-48c5-4c36-b54f-ff7af3e8d714 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/security-groups?id=d467c95d-fd2f-4e55-82e5-b5b76f6881e7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2247760 May 24 10:02:35.557846 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] For port 5479c5f8-df8c-4e32-9a54-3dff2b992272, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=5479c5f8-df8c-4e32-9a54-3dff2b992272,segment=NetworkSegment(0af48d33-b0f2-492b-b676-993923c0f33a),segment_id=0af48d33-b0f2-492b-b676-993923c0f33a)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:35.558003 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:35.735665 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Calling delete_port for 5479c5f8-df8c-4e32-9a54-3dff2b992272 owned by compute:nova {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:02:35.998620 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c1929e14-3a56-4f33-93b3-87badfdd2738 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:36.038554 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-da8e8723-5112-4f14-a353-bc1455c197a1 req-f105419d-4037-42d5-8799-5f2bc794eef5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:36.046539 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-da8e8723-5112-4f14-a353-bc1455c197a1 req-f105419d-4037-42d5-8799-5f2bc794eef5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2301016 May 24 10:02:36.108761 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-da8e8723-5112-4f14-a353-bc1455c197a1 req-4058ad58-95ff-4267-8516-213300cf6180 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0574236 May 24 10:02:36.184452 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8596653a-b70e-441e-86a1-fa5ec105fe1b req-8f313d0a-4ab2-45e2-a2a6-3de5f41460ee tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:36.195678 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8596653a-b70e-441e-86a1-fa5ec105fe1b req-8f313d0a-4ab2-45e2-a2a6-3de5f41460ee tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1309619 May 24 10:02:36.270512 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8596653a-b70e-441e-86a1-fa5ec105fe1b req-849502dd-4a3c-412f-9a6b-402055f425d6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0736654 May 24 10:02:36.272803 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:36.329863 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:36.335890 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8e0db0b7-922a-4853-a9ae-790bb684ad79 tempest-DeleteServersTestJSON-1347228157 tempest-DeleteServersTestJSON-1347228157-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:36.351584 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['5479c5f8-df8c-4e32-9a54-3dff2b992272'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:02:36.392399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Pushing event deleted for resources: {'Port': ['ID=5479c5f8-df8c-4e32-9a54-3dff2b992272,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:36.401579 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:36.499512 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6be8244c-9f55-445c-ba20-4b3510b4a6e9 req-55ac711a-e9ac-47cb-abda-bce2d35bc5fb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:36.505570 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6be8244c-9f55-445c-ba20-4b3510b4a6e9 req-55ac711a-e9ac-47cb-abda-bce2d35bc5fb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2172847 May 24 10:02:36.508293 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] No DVR routers for this DVR port 5479c5f8-df8c-4e32-9a54-3dff2b992272 on host hv-cinder-79123 {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:02:36.511466 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:36.526887 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'name': 'network-vif-deleted', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:36.561756 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6be8244c-9f55-445c-ba20-4b3510b4a6e9 req-cb3782ce-096c-4bd8-8773-dd186447b45b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0521796 May 24 10:02:36.604102 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5b09368d-88d6-4178-8690-92edbf156805', 'name': 'network-vif-deleted', 'tag': '5479c5f8-df8c-4e32-9a54-3dff2b992272', 'status': 'completed', 'code': 200} May 24 10:02:36.722209 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f980f9d4-ce0f-44cf-9057-2f1474ca6e8e req-fa5ea693-edc8-45ee-853d-a9f27fd5c9f4 tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b09368d-88d6-4178-8690-92edbf156805 HTTP/1.1" status: 200 len: 210 time: 0.0434403 May 24 10:02:36.743152 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-231252d3-b93f-4f68-9116-95b2d07145a7 req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] 10.222.0.95 "DELETE /v2.0/ports/5479c5f8-df8c-4e32-9a54-3dff2b992272 HTTP/1.1" status: 204 len: 173 time: 1.8032889 May 24 10:02:36.929013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55118) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:37.088690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Request body: {'network': {'name': 'tempest-VolumesSnapshotTestJSON-214993601-network', 'tenant_id': '0fa87500244642ee805dba47191bc7e6'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:37.096606 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:37.112513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Usage tracker for resource:network and tenant:0fa87500244642ee805dba47191bc7e6 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:37.117018 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:37.150833 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:37.176089 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.232904 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Made reservation on behalf of 0fa87500244642ee805dba47191bc7e6 for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:37.235429 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.259065 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.298048 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:37.310726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Usage tracker for resource:security_group_rule and tenant:0fa87500244642ee805dba47191bc7e6 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:37.315122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55130) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:37.318431 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:37.342185 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:37.500168 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63e16efa-5de3-438d-b9d4-ee04e980606d req-0a3e3820-0fb9-4bb3-81b9-e43dfa0b12ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:37.511960 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-63e16efa-5de3-438d-b9d4-ee04e980606d req-0a3e3820-0fb9-4bb3-81b9-e43dfa0b12ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2387698 May 24 10:02:37.582931 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.613942 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-63e16efa-5de3-438d-b9d4-ee04e980606d req-378a87f0-f51b-4e38-a75c-91fa4b1e07e3 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1007543 May 24 10:02:37.640042 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:37.651361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Usage tracker for resource:security_group_rule and tenant:003554ec229f48fbb905d9b97d667bc2 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:37.663658 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:37.676125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:37.693039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.715278 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85282006-f5dd-4bd9-8aae-2baba425f439 req-b3bcc82f-f0d5-49ca-960d-fe4711f2dc8f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:37.716790 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-85282006-f5dd-4bd9-8aae-2baba425f439 req-b3bcc82f-f0d5-49ca-960d-fe4711f2dc8f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.2868350 May 24 10:02:37.775136 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.782227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9600a6d4-3f7f-4714-9202-2c81a8987a5a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:37.804836 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-85282006-f5dd-4bd9-8aae-2baba425f439 req-3a6839f8-e297-435b-b226-8ecbf8423a2a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0790186 May 24 10:02:37.826909 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 63} {{(pid=109113) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:02:37.863765 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.864123 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Added segment 3ddab603-0e7a-4025-8390-7039b3ef3fd1 of type vlan for network 2852c3dc-6508-45a7-9576-a36787475d18 May 24 10:02:37.865470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:37.877378 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6272382-774c-4c1c-90ea-d794194581a0 req-dbdf1dd2-6174-4d35-9a5d-6da40289d4b5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:37.878960 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c6272382-774c-4c1c-90ea-d794194581a0 req-dbdf1dd2-6174-4d35-9a5d-6da40289d4b5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1294580 May 24 10:02:37.887503 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:37.959978 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c6272382-774c-4c1c-90ea-d794194581a0 req-291f6cdb-120f-4504-afc8-b39e7d649f5f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0677011 May 24 10:02:37.968688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:37.990347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.033010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.045228 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e35e665b-04d8-4b42-94ae-3e743b24f54d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:38.073597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.090384 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:38.091080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.111975 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=003554ec229f48fbb905d9b97d667bc2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7952342 May 24 10:02:38.114880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 55146) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:38.125014 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=321ed352-027b-4321-bfdf-39f96bc28323,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:38.147466 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:38.148072 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.047s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:38.251232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.267073 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Pushing event updated for resources: {'Network': ['ID=2852c3dc-6508-45a7-9576-a36787475d18,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:38.277954 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 1.3478253 May 24 10:02:38.280884 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:38.281470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55150) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:38.281951 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 2852c3dc-6508-45a7-9576-a36787475d18 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:02:38.298729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Request body: {'subnet': {'network_id': '2852c3dc-6508-45a7-9576-a36787475d18', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesSnapshotTestJSON-214993601-subnet', 'tenant_id': '0fa87500244642ee805dba47191bc7e6', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:38.319574 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:38.325521 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Usage tracker for resource:subnet and tenant:0fa87500244642ee805dba47191bc7e6 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:38.328767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:38.335308 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:38.340498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.410122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Made reservation on behalf of 0fa87500244642ee805dba47191bc7e6 for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:38.413572 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-128981363'] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.518535 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.525588 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-48a53a03-ca76-4397-927e-b53e72b76432 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:38.565713 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=321ed352-027b-4321-bfdf-39f96bc28323,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:38.585995 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:38.608064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.611449 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/321ed352-027b-4321-bfdf-39f96bc28323 HTTP/1.1" status: 204 len: 168 time: 0.4925580 May 24 10:02:38.733379 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:38.999468 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a181065-27ce-4705-804f-1515b087fc1b req-af90c7ba-587e-43c7-ac59-806ae51d82b0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:39.007501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6a181065-27ce-4705-804f-1515b087fc1b req-af90c7ba-587e-43c7-ac59-806ae51d82b0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1735842 May 24 10:02:39.146885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7d9f16b-cbbb-4147-8011-e796ae231086 req-717569db-19bf-4cbd-a3ed-a5000627c6bf tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:39.148537 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e7d9f16b-cbbb-4147-8011-e796ae231086 req-717569db-19bf-4cbd-a3ed-a5000627c6bf tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1909761 May 24 10:02:39.163516 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6a181065-27ce-4705-804f-1515b087fc1b req-1da0a237-39ee-4dc7-97bc-4faaab1ab05b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1546080 May 24 10:02:39.255329 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e7d9f16b-cbbb-4147-8011-e796ae231086 req-1c91d786-adb3-4448-a114-78b21409eb3a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0977497 May 24 10:02:39.364343 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-129003893', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-128981349'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:39.368399 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:39.448880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 55180) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:39.469421 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:39.486257 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:39.492574 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Usage tracker for resource:security_group_rule and tenant:088b09a23a1241519b4c93a8ec7617c8 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:39.498610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:39.504639 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:39.558341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Pushing event updated for resources: {'Subnet': ['ID=9ee9ac65-ecfe-47d4-bbac-4561dabf43e6,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:39.563743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.195s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:39.575594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Checking segment: {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:39.575883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Network hostable DHCP agents. Network: 2852c3dc-6508-45a7-9576-a36787475d18, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:02:39.646304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90914231-6fc9-4bc7-bc9e-cb1a7d829d2a req-1d65f371-63c3-4049-ab97-a5160968df8e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:39.659777 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-90914231-6fc9-4bc7-bc9e-cb1a7d829d2a req-1d65f371-63c3-4049-ab97-a5160968df8e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2558355 May 24 10:02:39.665298 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 5479c5f8-df8c-4e32-9a54-3dff2b992272 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:02:39.695904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:39.718163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Network 2852c3dc-6508-45a7-9576-a36787475d18 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:02:39.742833 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 5479c5f8-df8c-4e32-9a54-3dff2b992272 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:02:39.743326 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 5479c5f8-df8c-4e32-9a54-3dff2b992272 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:02:39.766607 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-90914231-6fc9-4bc7-bc9e-cb1a7d829d2a req-f617ab0e-7e96-4e0d-9d4f-f9df998a1ee4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1052153 May 24 10:02:39.776126 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:39.787394 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:39.815653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 2852c3dc-6508-45a7-9576-a36787475d18) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:39.836404 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:39.837000 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:39.837989 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fef56456-868f-4e39-a9a9-d99cba10046b None None] Network 2852c3dc-6508-45a7-9576-a36787475d18 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:02:39.929889 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=088b09a23a1241519b4c93a8ec7617c8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4796162 May 24 10:02:39.932476 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55192) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:39.937919 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2a570968-4bd3-42e7-a10e-570029c4ebfb,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:39.956487 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.171s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:40.004459 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e59df962-76a6-456f-ad32-a66091caa0c1 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 1.7048504 May 24 10:02:40.017430 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:40.049587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:40.127334 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fb617f14-3b94-47de-be2e-e8269e42ee7a None None] Network 2852c3dc-6508-45a7-9576-a36787475d18 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:02:40.198770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:40.203441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-de430c08-4ddf-4ab1-9d8b-086bc8c11be3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:40.245327 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2a570968-4bd3-42e7-a10e-570029c4ebfb,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:40.270578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:40.279456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:40.281959 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/2a570968-4bd3-42e7-a10e-570029c4ebfb HTTP/1.1" status: 204 len: 168 time: 0.3484528 May 24 10:02:40.577219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07c6a5b9-5904-46c2-a2f3-f4aff301fc70 req-e4031f16-8f39-4ee6-9c00-3c76453b213a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:40.588260 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-07c6a5b9-5904-46c2-a2f3-f4aff301fc70 req-e4031f16-8f39-4ee6-9c00-3c76453b213a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1703150 May 24 10:02:40.617359 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f47e7c0b-92cb-49b7-a0d7-d314a8d01de1 req-6ca25801-b5e6-45c5-83d0-7d437a9ebb50 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:40.635687 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f47e7c0b-92cb-49b7-a0d7-d314a8d01de1 req-6ca25801-b5e6-45c5-83d0-7d437a9ebb50 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1235654 May 24 10:02:40.649308 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-07c6a5b9-5904-46c2-a2f3-f4aff301fc70 req-ecf55b35-673d-4ce3-840e-c167775f54c0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0575142 May 24 10:02:40.721244 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f47e7c0b-92cb-49b7-a0d7-d314a8d01de1 req-17424619-6c65-42c3-9edf-45fcd9563af2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0842810 May 24 10:02:41.013009 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-341e0a0a-a655-427b-83c3-6f9662743e8a req-05ade2c8-17a0-473e-8e6b-ba5001470d49 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:41.020466 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-341e0a0a-a655-427b-83c3-6f9662743e8a req-05ade2c8-17a0-473e-8e6b-ba5001470d49 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0974977 May 24 10:02:41.073461 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-341e0a0a-a655-427b-83c3-6f9662743e8a req-c7917874-b4f8-49e4-a6f0-cb13be858c32 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0507965 May 24 10:02:41.929216 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-398a4c4d-3292-4ad6-b9c2-5e97ecb863c6 req-24b7161e-8155-4fbe-858c-389b6735f8dc tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:41.934920 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-398a4c4d-3292-4ad6-b9c2-5e97ecb863c6 req-24b7161e-8155-4fbe-858c-389b6735f8dc tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1000197 May 24 10:02:41.960283 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41af8f0b-2625-4469-b0bb-9c375d3f4b09 req-1988c94b-a6dd-4b91-ab79-52fbce64467b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:41.961787 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-41af8f0b-2625-4469-b0bb-9c375d3f4b09 req-1988c94b-a6dd-4b91-ab79-52fbce64467b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1848247 May 24 10:02:42.020242 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-398a4c4d-3292-4ad6-b9c2-5e97ecb863c6 req-48d72b97-0249-464a-b303-e7396a32692a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0747793 May 24 10:02:42.041668 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-41af8f0b-2625-4469-b0bb-9c375d3f4b09 req-bbe051de-7aae-4aa6-b169-de4a0c886ac5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0720363 May 24 10:02:42.388215 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60ae2167-2a44-4302-9582-d88c42955a92 req-205d18e4-a73e-4755-9b4c-5364a2eba3ea tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:42.389844 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-60ae2167-2a44-4302-9582-d88c42955a92 req-205d18e4-a73e-4755-9b4c-5364a2eba3ea tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1095285 May 24 10:02:42.447824 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-60ae2167-2a44-4302-9582-d88c42955a92 req-0bb886f2-6e0a-4b5b-85a8-bc67982faf2a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0518565 May 24 10:02:43.278384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db8dda38-f84e-4c0a-a068-f7f1cade4ad7 req-0b15dd8d-53cc-4b24-8bbc-6db5374d2c53 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:43.279924 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-db8dda38-f84e-4c0a-a068-f7f1cade4ad7 req-0b15dd8d-53cc-4b24-8bbc-6db5374d2c53 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0944235 May 24 10:02:43.304474 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea0c602e-db7b-45b5-bdde-0c6fbfee9f86 req-f21ab1be-569c-463f-9076-90cee43c845a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:43.314021 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ea0c602e-db7b-45b5-bdde-0c6fbfee9f86 req-f21ab1be-569c-463f-9076-90cee43c845a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1230962 May 24 10:02:43.351771 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-db8dda38-f84e-4c0a-a068-f7f1cade4ad7 req-93d53387-2c5a-4141-b68c-1d8db1591a15 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0629756 May 24 10:02:43.385880 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ea0c602e-db7b-45b5-bdde-0c6fbfee9f86 req-82c37d74-d2df-42fe-b08c-06e4abf65d97 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0705810 May 24 10:02:43.439822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 55276) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:43.679048 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:43.698871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:43.704675 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Usage tracker for resource:security_group_rule and tenant:8750461cd0a74441bdd726e44cf85db2 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:43.708056 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:43.718247 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:43.760261 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46d293fd-e815-4fc4-8571-c39e3d2539bc req-71b8d369-7e15-4f4a-9397-0fcb158dbd8d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:43.761829 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46d293fd-e815-4fc4-8571-c39e3d2539bc req-71b8d369-7e15-4f4a-9397-0fcb158dbd8d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1578987 May 24 10:02:43.830182 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46d293fd-e815-4fc4-8571-c39e3d2539bc req-177330f6-4ed7-42e6-b2ca-a38c04bbe8a2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0618999 May 24 10:02:43.871395 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:43.980849 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:43.987757 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2b0a04ea-6d0e-418d-b3bf-68537321d664 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:44.031794 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=8750461cd0a74441bdd726e44cf85db2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5900605 May 24 10:02:44.032783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0b8e6b4a-61b5-4dd6-a191-28fc7a4556f2,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:44.034524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55312) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:44.036799 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:44.122403 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:44.218096 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:44.305514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-e8455b7f-2b0b-4e12-ae01-9a05f36c7ba6 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:44.306725 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-e8455b7f-2b0b-4e12-ae01-9a05f36c7ba6 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:network {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:44.411595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:44.414315 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-e8455b7f-2b0b-4e12-ae01-9a05f36c7ba6 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:network {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:44.415938 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-e8455b7f-2b0b-4e12-ae01-9a05f36c7ba6 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/networks?id=2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 200 len: 813 time: 0.3142567 May 24 10:02:44.418432 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8252c067-0083-483d-af4d-20ebae296bf2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:44.450628 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0b8e6b4a-61b5-4dd6-a191-28fc7a4556f2,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:44.455778 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:44.464199 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-10c5e145-4ac3-493a-900f-2bb958f0caad tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/quotas/0fa87500244642ee805dba47191bc7e6 HTTP/1.1" status: 200 len: 375 time: 0.0364683 May 24 10:02:44.467338 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:44.471309 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/0b8e6b4a-61b5-4dd6-a191-28fc7a4556f2 HTTP/1.1" status: 204 len: 168 time: 0.4351258 May 24 10:02:44.520962 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-2d32fa71-4f31-4ade-8cd1-1576af011bbb tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0530331 May 24 10:02:44.667986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6161ef94-0d7a-4519-9cf4-fc0ea2d5bfd3 req-dc816004-7956-4ae0-9be9-199555ae8450 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:44.669730 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6161ef94-0d7a-4519-9cf4-fc0ea2d5bfd3 req-dc816004-7956-4ae0-9be9-199555ae8450 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1581464 May 24 10:02:44.679593 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-0882a8a7-efe7-4b9b-acc8-3c59cbc1cf64 service neutron] 10.222.0.67 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1526153 May 24 10:02:44.728417 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6161ef94-0d7a-4519-9cf4-fc0ea2d5bfd3 req-773a03ca-330c-4cbd-a200-93806f23c5c3 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0522270 May 24 10:02:44.832918 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-7d142700-df3d-48dc-b3b8-e1f1270d6844 service neutron] 10.222.0.67 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1474276 May 24 10:02:44.928748 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22e4bb6a-3416-4fca-939b-3f834962b868 req-05446853-7dd2-4f0e-b786-25f0213e2670 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:44.930627 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-22e4bb6a-3416-4fca-939b-3f834962b868 req-05446853-7dd2-4f0e-b786-25f0213e2670 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0956035 May 24 10:02:44.985788 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-22e4bb6a-3416-4fca-939b-3f834962b868 req-20a784a9-dee9-4e3e-9756-f28329091957 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0484176 May 24 10:02:45.183674 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22c977a7-a922-446b-b628-4e00bba43804 req-1be336a7-5942-4d98-81fc-b89422724605 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.0458336 May 24 10:02:45.197441 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de4a3576-160b-4db7-903b-3d0505840a05 req-07aa2dee-92dc-49a6-b73b-ccb5a05e3d97 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:45.199227 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-de4a3576-160b-4db7-903b-3d0505840a05 req-07aa2dee-92dc-49a6-b73b-ccb5a05e3d97 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1169822 May 24 10:02:45.251043 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-de4a3576-160b-4db7-903b-3d0505840a05 req-07438358-b19a-4889-b62b-c6741814ca9f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0452275 May 24 10:02:45.923886 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79a4dacb-e07f-4077-be12-6f33822fb4d1 req-78ac2f2c-88d2-4903-9c75-a824311c585f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:45.925493 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-79a4dacb-e07f-4077-be12-6f33822fb4d1 req-78ac2f2c-88d2-4903-9c75-a824311c585f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0864928 May 24 10:02:45.973503 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-79a4dacb-e07f-4077-be12-6f33822fb4d1 req-d9273eec-e381-452e-b0e6-26cd354caee6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0406442 May 24 10:02:46.174853 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a438aad2-fa5b-4774-a9b3-3a93dfbf31d3 req-be05df27-4d97-45be-8869-e65ab9f94c78 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:46.176306 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a438aad2-fa5b-4774-a9b3-3a93dfbf31d3 req-be05df27-4d97-45be-8869-e65ab9f94c78 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0792811 May 24 10:02:46.224157 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a438aad2-fa5b-4774-a9b3-3a93dfbf31d3 req-74fdde83-e6eb-4e9b-9cb4-6cae0797ebce tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414257 May 24 10:02:46.286335 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4d6432ae-1d2a-444f-9a2c-445de218803e req-207a25ff-b866-4b6e-bbb7-353092f38a9c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.0450351 May 24 10:02:46.463739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3abffab-bf16-412e-a2d2-91d85e30a2d5 req-3e03b417-5a12-434f-b892-e61a4af4a8d0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:46.463739 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b3abffab-bf16-412e-a2d2-91d85e30a2d5 req-3e03b417-5a12-434f-b892-e61a4af4a8d0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.0873618 May 24 10:02:46.508858 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b3abffab-bf16-412e-a2d2-91d85e30a2d5 req-351d7582-7737-4c73-9eb2-a05be0871613 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0405736 May 24 10:02:47.183062 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc703145-c373-47a6-9099-b8d148be2f24 req-fa37899e-5e21-4581-b5bd-29b52ab3202e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.184523 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-dc703145-c373-47a6-9099-b8d148be2f24 req-fa37899e-5e21-4581-b5bd-29b52ab3202e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0842409 May 24 10:02:47.229337 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-dc703145-c373-47a6-9099-b8d148be2f24 req-ccf2eff6-b29f-4a51-9c72-da83faa03f03 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390029 May 24 10:02:47.479997 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-15b55cb0-5d47-49d1-b50a-58846a37eaa8 req-48c3bea5-a1fc-4704-bbf1-3a7bab84def5 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.0547607 May 24 10:02:47.529263 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b8c7545-9323-4fdc-be0d-7c7f63fefd75 req-5f0fd8fb-4f68-4b33-aa4d-a3396687b847 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.530735 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0b8c7545-9323-4fdc-be0d-7c7f63fefd75 req-5f0fd8fb-4f68-4b33-aa4d-a3396687b847 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1775765 May 24 10:02:47.594080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-e909f027-2d5c-4b2d-9a4c-1c4037f748b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.596196 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-e909f027-2d5c-4b2d-9a4c-1c4037f748b6 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.0933263 May 24 10:02:47.653675 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0b8c7545-9323-4fdc-be0d-7c7f63fefd75 req-ada94ef4-07fb-4540-9e64-faa9536981d8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1071632 May 24 10:02:47.663057 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-3e30b713-9653-499c-bfbb-654f689228a5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.673513 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-3e30b713-9653-499c-bfbb-654f689228a5 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1099 time: 0.2216949 May 24 10:02:47.843241 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-1bdb26b8-2998-40d7-9486-3380988f22d0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.844605 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-85d487db-f25b-438c-a05c-c00b35c4ac13 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.844994 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-1bdb26b8-2998-40d7-9486-3380988f22d0 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.2194676 May 24 10:02:47.846751 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-85d487db-f25b-438c-a05c-c00b35c4ac13 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.1662583 May 24 10:02:47.853675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d56d22d-85f9-4c60-898a-80491a5d0d14 req-b51c8e38-9244-4b8d-9eed-cbc8d0af554e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:47.855159 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1d56d22d-85f9-4c60-898a-80491a5d0d14 req-b51c8e38-9244-4b8d-9eed-cbc8d0af554e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.2032433 May 24 10:02:47.990015 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1d56d22d-85f9-4c60-898a-80491a5d0d14 req-d123a824-a844-4525-b709-6f5caf9983db tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1282585 May 24 10:02:48.014808 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-02c188e3-10e0-4df2-a02b-97e62a0628f0 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1000 time: 0.1283906 May 24 10:02:48.045351 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-f8eeb62e-3d9c-4280-8f8f-7557bc0876c1 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 1001 time: 0.1597056 May 24 10:02:48.138265 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-38f9319a-305b-41fe-a5c5-52a0899c4978 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:48.140046 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-38f9319a-305b-41fe-a5c5-52a0899c4978 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0880883 May 24 10:02:48.222422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-eae0240e-fa0a-4a59-8e32-5b17772510c1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:48.223879 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-eae0240e-fa0a-4a59-8e32-5b17772510c1 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.1968107 May 24 10:02:48.239588 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-ba24e7b2-8f39-40f4-8f38-7624736fc723 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:48.240977 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-ba24e7b2-8f39-40f4-8f38-7624736fc723 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0942111 May 24 10:02:48.347502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-3c67b451-38fd-4abe-8aad-c846ee64199c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:48.349086 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-3c67b451-38fd-4abe-8aad-c846ee64199c service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1169426 May 24 10:02:48.416627 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-d5c1f9a6-8b16-4c41-906e-737f651a917c service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1653433 May 24 10:02:48.590772 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-4078f675-9474-46ed-a6a4-e7011989b082 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2314129 May 24 10:02:48.603333 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b0d7c8fd-22a5-4ac1-b1c8-f6e142ac27e9 req-1b87a459-0137-41fd-bd92-7af5346a0ead service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1746006 May 24 10:02:48.734731 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a557505b-b741-4c50-8369-4817145284cb req-83e9de91-9e24-4541-b250-3945681ab2c7 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:48.739181 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a557505b-b741-4c50-8369-4817145284cb req-83e9de91-9e24-4541-b250-3945681ab2c7 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1346536 May 24 10:02:48.816598 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a557505b-b741-4c50-8369-4817145284cb req-c7e3fc74-a53f-41bc-ba69-0bcccba04bd9 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0678408 May 24 10:02:48.996044 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-dfad190e-89f8-4e19-9b64-ba542707a845 req-eda2cc06-e11d-448d-93c9-6c9943876c36 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.2937589 May 24 10:02:49.084233 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45f5d9eb-7673-4cc8-9aa3-d342b002736f req-ee2b67c3-2020-415e-ae64-a5625d99c58a service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4832339 May 24 10:02:49.317757 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f31cdfca-90cf-46e1-bf94-a885de21af4c req-3bd6423a-c9b3-415d-855a-165bb0d812b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:49.325920 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f31cdfca-90cf-46e1-bf94-a885de21af4c req-3bd6423a-c9b3-415d-855a-165bb0d812b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 946 time: 0.1629329 May 24 10:02:49.424148 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f31cdfca-90cf-46e1-bf94-a885de21af4c req-c8c3aa42-d6d7-46d3-9321-44851c9095b7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0901055 May 24 10:02:49.501597 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-1225d6f1-e056-4ddc-ba75-1aebf89c7779 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:49.503069 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-1225d6f1-e056-4ddc-ba75-1aebf89c7779 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "GET /v2.0/networks?id=2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 200 len: 813 time: 0.4446378 May 24 10:02:49.516264 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-42d140fd-5841-40bf-bd1d-4e612c094adf req-4fb43657-fefc-4e9e-a064-0cd0fb8e2b46 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:49.524417 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-42d140fd-5841-40bf-bd1d-4e612c094adf req-4fb43657-fefc-4e9e-a064-0cd0fb8e2b46 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.5740075 May 24 10:02:49.536236 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Request body: {'port': {'device_id': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18', 'admin_state_up': True, 'tenant_id': '0fa87500244642ee805dba47191bc7e6'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:49.690372 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-42d140fd-5841-40bf-bd1d-4e612c094adf req-a11bec00-e343-4fe9-9f80-ac76149b0019 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1643829 May 24 10:02:49.724985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:02:49.801478 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:02:49.806014 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Usage tracker for resource:port and tenant:0fa87500244642ee805dba47191bc7e6 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:02:49.809451 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Quota usage for port was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:02:49.832611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:02:49.908226 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Made reservation on behalf of 0fa87500244642ee805dba47191bc7e6 for: {'port': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:02:49.911675 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:49.916797 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:49.960077 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:02:49.961355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:50.044259 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:50.050505 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:50.105425 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'code': 200} May 24 10:02:50.165845 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:50.174107 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=8']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:50.189414 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.192241 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.196106 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.197911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.198402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.199110 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.200548 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.201629 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.212362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.212362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.212362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.212362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:02:50.226624 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:50.230649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Allocated IP 172.20.0.14 (2852c3dc-6508-45a7-9576-a36787475d18/9ee9ac65-ecfe-47d4-bbac-4561dabf43e6/855e6647-516e-462d-8d07-0f5cf828da14) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:02:50.247244 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e368250d-0b29-4a6e-a78c-04eea6c189be req-15d3e53c-d2e1-4cb6-87ea-42329b6f76ce tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:50.256352 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e368250d-0b29-4a6e-a78c-04eea6c189be req-15d3e53c-d2e1-4cb6-87ea-42329b6f76ce tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2167706 May 24 10:02:50.307176 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e368250d-0b29-4a6e-a78c-04eea6c189be req-90c6ccfe-6e3f-44a1-ac85-592ed2b9ad46 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0499592 May 24 10:02:50.418442 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-be77369a-9deb-423a-9ef7-dcb2fc9d50ff req-6413c6a2-235e-4d9c-83b2-cb6eabd49c0f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.0676832 May 24 10:02:50.614654 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:50.669244 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03319d20-dd5e-473d-a1ac-fd42eb960a37 req-969ff8da-0ea0-4464-853b-717fe63814a0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:50.670765 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-03319d20-dd5e-473d-a1ac-fd42eb960a37 req-969ff8da-0ea0-4464-853b-717fe63814a0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0911968 May 24 10:02:50.731499 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:50.736616 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-03319d20-dd5e-473d-a1ac-fd42eb960a37 req-dbd4d7ba-a610-47f2-aa11-59744a8e324b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0578227 May 24 10:02:50.739711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-373d6a3c-8268-446b-b643-62ed54450e0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:50.833924 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Network 2852c3dc-6508-45a7-9576-a36787475d18 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:02:50.837405 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:02:50.840275 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:02:50.847978 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:50.848569 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:50.870046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Pushing event updated for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:50.873281 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:50.976812 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d6b038e-e325-40d6-971c-16711c52ce0e req-a99461bf-8bea-4b53-9ef7-fd55ec92b362 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:50.978986 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5d6b038e-e325-40d6-971c-16711c52ce0e req-a99461bf-8bea-4b53-9ef7-fd55ec92b362 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1003697 May 24 10:02:51.030218 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5d6b038e-e325-40d6-971c-16711c52ce0e req-fc7131dd-8358-4346-9b8f-603b6b830b61 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448227 May 24 10:02:51.031786 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.5039201 May 24 10:02:51.582020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-929636a0-6a73-4212-8b2c-08c5889fc4df req-71c9716f-1c67-4718-974a-d3c6fceab98b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:51.589327 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-929636a0-6a73-4212-8b2c-08c5889fc4df req-71c9716f-1c67-4718-974a-d3c6fceab98b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1309197 May 24 10:02:51.639524 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c19bc34b-ee06-4d6d-ba4d-7115f8ffe141 req-f2a6668f-d595-459d-97cd-937b09230f5d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:51.640321 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c19bc34b-ee06-4d6d-ba4d-7115f8ffe141 req-f2a6668f-d595-459d-97cd-937b09230f5d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:51.649366 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-929636a0-6a73-4212-8b2c-08c5889fc4df req-ce9fd787-589e-46c2-8489-4b68bbed78b1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0590165 May 24 10:02:51.752871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c19bc34b-ee06-4d6d-ba4d-7115f8ffe141 req-f2a6668f-d595-459d-97cd-937b09230f5d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:51.754544 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c19bc34b-ee06-4d6d-ba4d-7115f8ffe141 req-f2a6668f-d595-459d-97cd-937b09230f5d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 932 time: 0.2017384 May 24 10:02:51.927617 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c19bc34b-ee06-4d6d-ba4d-7115f8ffe141 req-9000ad67-e236-473b-b566-dad9f6e5b677 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1615744 May 24 10:02:52.166656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036ba71a-dd14-457f-ab27-17e216702d08 req-777acd26-cc31-4642-bfab-2f720290700d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:52.174600 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-036ba71a-dd14-457f-ab27-17e216702d08 req-777acd26-cc31-4642-bfab-2f720290700d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0967200 May 24 10:02:52.233610 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-036ba71a-dd14-457f-ab27-17e216702d08 req-10e70824-770d-44bf-b1f7-3f65641508f5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0580928 May 24 10:02:52.303707 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5ddd8e8-feff-4d67-a09a-ab4e42919ad2 req-9b6840fe-bcfb-4ec6-9a56-a94439653aff tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:52.303707 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c5ddd8e8-feff-4d67-a09a-ab4e42919ad2 req-9b6840fe-bcfb-4ec6-9a56-a94439653aff tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1073527 May 24 10:02:52.344299 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c5ddd8e8-feff-4d67-a09a-ab4e42919ad2 req-52fc2a07-288f-449a-a37a-7f32bbee3b87 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400002 May 24 10:02:52.888247 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82708fe2-6432-4bd5-ba47-24f7ce8a11b7 req-7ee41efe-dce3-401d-aa61-17050a4d2d57 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:52.889733 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-82708fe2-6432-4bd5-ba47-24f7ce8a11b7 req-7ee41efe-dce3-401d-aa61-17050a4d2d57 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1115139 May 24 10:02:52.947991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-82708fe2-6432-4bd5-ba47-24f7ce8a11b7 req-a00d30db-1881-4859-b723-f9b89fb51fa7 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0512609 May 24 10:02:53.262251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e789afdf-89aa-49af-ab70-7906d3171b11 req-1dd0885a-9890-4101-990d-6c0e147727df tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:53.263946 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e789afdf-89aa-49af-ab70-7906d3171b11 req-1dd0885a-9890-4101-990d-6c0e147727df tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 932 time: 0.1079783 May 24 10:02:53.370939 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e789afdf-89aa-49af-ab70-7906d3171b11 req-bd3aa632-e690-486c-8c71-30eb12de0044 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0866141 May 24 10:02:53.500945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b5ad1a4-55c6-4c36-81c8-d8787475458b req-8c2ec74d-08b3-43df-8d71-57e878504206 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:53.518447 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7b5ad1a4-55c6-4c36-81c8-d8787475458b req-8c2ec74d-08b3-43df-8d71-57e878504206 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1295133 May 24 10:02:53.646596 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7b5ad1a4-55c6-4c36-81c8-d8787475458b req-3aed7c34-f2c8-4e91-bf35-6b784233c38a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1271496 May 24 10:02:53.696113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5da4b894-c97c-43c3-8f8c-2ec82562fcb3 req-c5121bf4-598b-4eac-8c36-6e3b6089b9f5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:53.697678 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5da4b894-c97c-43c3-8f8c-2ec82562fcb3 req-c5121bf4-598b-4eac-8c36-6e3b6089b9f5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1523478 May 24 10:02:53.760195 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5da4b894-c97c-43c3-8f8c-2ec82562fcb3 req-3c07a56f-73be-4482-af9d-d85d71cf327c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0539167 May 24 10:02:53.801384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Request body: {'port': {'device_id': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:02:53.927359 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:54.173336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60530bdd-09ee-42ba-adff-122cdcae396b req-ba274260-10c3-468d-8c34-090c4f0a2298 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:54.182607 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-60530bdd-09ee-42ba-adff-122cdcae396b req-ba274260-10c3-468d-8c34-090c4f0a2298 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0994687 May 24 10:02:54.236138 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-60530bdd-09ee-42ba-adff-122cdcae396b req-198b9c9a-41f2-4c7f-97ea-67dde83fa7dc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0525324 May 24 10:02:54.366552 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 855e6647-516e-462d-8d07-0f5cf828da14 {{(pid=109112) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:02:54.376208 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:54.392903 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Transition to ACTIVE for port object 855e6647-516e-462d-8d07-0f5cf828da14 will not be triggered until provisioned by entity L2. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:02:54.455994 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:54.462293 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b4b7aecf-cd99-407e-aa60-18747411bfcd service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:54.764497 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Pushing event updated for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:54.801929 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.338s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:54.831482 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Attempting to bind port 855e6647-516e-462d-8d07-0f5cf828da14 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:02:54.831711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Attempting to bind port 855e6647-516e-462d-8d07-0f5cf828da14 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:02:54.832041 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Attempting to bind port 855e6647-516e-462d-8d07-0f5cf828da14 on network 2852c3dc-6508-45a7-9576-a36787475d18 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:54.843460 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-025a7962-9a6a-4757-8e10-ddccb1125f9d req-d2e1870d-b7a4-40d0-a6ea-4c67cd301891 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:54.850531 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-025a7962-9a6a-4757-8e10-ddccb1125f9d req-d2e1870d-b7a4-40d0-a6ea-4c67cd301891 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 944 time: 0.3415964 May 24 10:02:54.868825 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Port 855e6647-516e-462d-8d07-0f5cf828da14 on network 2852c3dc-6508-45a7-9576-a36787475d18 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:02:54.869299 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Attempting to bind port 855e6647-516e-462d-8d07-0f5cf828da14 on network 2852c3dc-6508-45a7-9576-a36787475d18 {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:02:54.881315 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 2, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:02:54.882046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Checking segment: {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:02:54.882046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Bound using segment: {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'} {{(pid=109112) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:02:54.882517 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.managers [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Bound port: 855e6647-516e-462d-8d07-0f5cf828da14, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'}}] {{(pid=109112) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:02:54.882837 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:54.974912 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-025a7962-9a6a-4757-8e10-ddccb1125f9d req-b32e09f6-4393-49f9-b425-9b47dfc759ee tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1221502 May 24 10:02:55.075503 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, cleared binding levels {{(pid=109112) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:02:55.080793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=855e6647-516e-462d-8d07-0f5cf828da14,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109112) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:02:55.119710 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df779680-c95a-4406-9f03-ec9d7ecf4267 req-ed5e4422-f54e-42e5-92f8-f30d96514eec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.121539 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-df779680-c95a-4406-9f03-ec9d7ecf4267 req-ed5e4422-f54e-42e5-92f8-f30d96514eec tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.2168291 May 24 10:02:55.164443 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:55.168752 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-77f97f96-8e17-43dd-b445-1f431c019dc2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:55.203335 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-df779680-c95a-4406-9f03-ec9d7ecf4267 req-6cae75eb-b240-4c72-9417-40493f2d0534 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0740547 May 24 10:02:55.306201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.306484 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:55.307756 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] 10.222.0.95 "PUT /v2.0/ports/855e6647-516e-462d-8d07-0f5cf828da14 HTTP/1.1" status: 200 len: 1094 time: 1.5109398 May 24 10:02:55.309557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:02:55.335362 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Pushing event updated for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=3']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:55.340261 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:55.365885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78bf9f6c-30b3-4be8-99e0-d95720fc02cf req-5c7e5c54-3fc2-4301-9d52-3d49aa51aa06 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.374014 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78bf9f6c-30b3-4be8-99e0-d95720fc02cf req-5c7e5c54-3fc2-4301-9d52-3d49aa51aa06 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1510801 May 24 10:02:55.524479 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78bf9f6c-30b3-4be8-99e0-d95720fc02cf req-9d0b2dcc-73c4-497a-b4cc-39198f3df31b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1414380 May 24 10:02:55.591684 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-5ee772aa-f2a5-4102-86b7-1a38f1fe9618 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.593953 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-5ee772aa-f2a5-4102-86b7-1a38f1fe9618 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 1097 time: 0.1710005 May 24 10:02:55.741333 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55772) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:55.750511 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-501a9dd3-fdd6-4ed7-b983-2b481c17b1d2 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=855e6647-516e-462d-8d07-0f5cf828da14 HTTP/1.1" status: 200 len: 217 time: 0.1490474 May 24 10:02:55.906251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-25bebd53-1f8c-4bd9-8492-f73f1a7a83fd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.907611 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-25bebd53-1f8c-4bd9-8492-f73f1a7a83fd service neutron] 10.222.0.95 "GET /v2.0/subnets?id=9ee9ac65-ecfe-47d4-bbac-4561dabf43e6 HTTP/1.1" status: 200 len: 853 time: 0.1454678 May 24 10:02:55.972370 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-87636146-fd87-4b7e-86d7-f9cadd1af2ae service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=2852c3dc-6508-45a7-9576-a36787475d18&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0549870 May 24 10:02:55.993419 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ba9a76f-af62-416f-9b64-bd48f617296a req-ec29be76-5e40-4af4-8011-e44c09ed44ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:55.997812 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9ba9a76f-af62-416f-9b64-bd48f617296a req-ec29be76-5e40-4af4-8011-e44c09ed44ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1844456 May 24 10:02:56.054683 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-ca8ffa73-51f6-4bb0-92ca-51e26aeb4ccd admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=efa03a8a310f4bf9ba9e8166f64c9983&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3125393 May 24 10:02:56.057645 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55792) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:56.130681 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:56.134151 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9ba9a76f-af62-416f-9b64-bd48f617296a req-9f213fd8-0f61-4eb9-8cc1-8cd7d1d05bc1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1031008 May 24 10:02:56.191157 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:56.295013 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14', 'status': 'completed', 'code': 200} May 24 10:02:56.305915 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-d5f303dd-d65b-4c60-9c50-79794f5e8edf service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3086061 May 24 10:02:56.472575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:56.477661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-825bc559-fb86-4e65-8a8f-8f6a4da6fbc5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:02:56.495901 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1e5cc604-d960-445e-ad7a-c0244613b030,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:02:56.498403 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:02:56.564466 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a73fa4bb-dc69-415c-88f5-40aa9a00bba3 req-31669ef0-e6a2-4ed4-ae79-d6fc798805b3 service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2520294 May 24 10:02:56.590294 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84e6f0e1-2d60-4985-9e44-d6c5162bb050 req-1beb9944-191e-4fcb-a666-9039d543253b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:56.591881 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-84e6f0e1-2d60-4985-9e44-d6c5162bb050 req-1beb9944-191e-4fcb-a666-9039d543253b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 944 time: 0.1346624 May 24 10:02:56.603259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:02:56.604624 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-ff73c906-8d0c-49ed-9b70-2d2333a3ce68 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/1e5cc604-d960-445e-ad7a-c0244613b030 HTTP/1.1" status: 204 len: 168 time: 0.5453460 May 24 10:02:56.695054 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-84e6f0e1-2d60-4985-9e44-d6c5162bb050 req-db96791c-773e-423c-9219-7b72853fa639 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0964561 May 24 10:02:56.863886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-9eeef6ec-0d2e-4d68-83dd-387f907a79e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:56.865774 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-9eeef6ec-0d2e-4d68-83dd-387f907a79e7 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 1097 time: 0.1659153 May 24 10:02:57.155430 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-1c2db274-155c-4ef2-8586-95fa87419d59 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:57.155430 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-1c2db274-155c-4ef2-8586-95fa87419d59 service neutron] 10.222.0.95 "GET /v2.0/networks?id=2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 200 len: 913 time: 0.2798491 May 24 10:02:57.173322 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46f23a46-bfc9-459d-9452-aee5aa894980 req-c64520b7-8ad8-4c94-a06f-e657648c29b7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:57.173322 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46f23a46-bfc9-459d-9452-aee5aa894980 req-c64520b7-8ad8-4c94-a06f-e657648c29b7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.2289531 May 24 10:02:57.407807 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-04360cd6-992e-496a-afb1-9efad63e911a service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=855e6647-516e-462d-8d07-0f5cf828da14 HTTP/1.1" status: 200 len: 217 time: 0.2333887 May 24 10:02:57.476979 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46f23a46-bfc9-459d-9452-aee5aa894980 req-914087a6-9938-408e-93ed-71f5357bb7fa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0995126 May 24 10:02:57.510909 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-33b11028-40a4-4221-9273-7cc40288a938 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:57.512583 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-33b11028-40a4-4221-9273-7cc40288a938 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=9ee9ac65-ecfe-47d4-bbac-4561dabf43e6 HTTP/1.1" status: 200 len: 853 time: 0.0915041 May 24 10:02:57.575142 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b122037-8529-423d-b190-83136871c381 req-aec515c0-b491-4a8c-a8e5-8b3466a8ae36 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:57.577620 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-ac41fce8-1d5f-4cf7-9d26-8e34cd23a312 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=2852c3dc-6508-45a7-9576-a36787475d18&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0576360 May 24 10:02:57.582791 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0b122037-8529-423d-b190-83136871c381 req-aec515c0-b491-4a8c-a8e5-8b3466a8ae36 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1731067 May 24 10:02:57.675865 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0b122037-8529-423d-b190-83136871c381 req-c376f760-c932-45db-97b0-a3bbd3b62e91 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0920765 May 24 10:02:57.804786 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:02:57.898241 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-3a87fa61-6f89-4ba0-9a9d-8e233ca5c098 service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3126252 May 24 10:02:58.139381 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5adf6289-c42a-40e0-8a9c-c55c59fdc5a8 req-50dd1182-e653-4e48-9c88-b161ccfb708e tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:58.163179 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5adf6289-c42a-40e0-8a9c-c55c59fdc5a8 req-50dd1182-e653-4e48-9c88-b161ccfb708e tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 944 time: 0.2670245 May 24 10:02:58.249666 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-5adf6289-c42a-40e0-8a9c-c55c59fdc5a8 req-29247948-d742-4296-8dbc-6f36f8f9eea5 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:02:58.337797 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-5adf6289-c42a-40e0-8a9c-c55c59fdc5a8 req-29247948-d742-4296-8dbc-6f36f8f9eea5 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:02:58.340551 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5adf6289-c42a-40e0-8a9c-c55c59fdc5a8 req-29247948-d742-4296-8dbc-6f36f8f9eea5 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1700909 May 24 10:02:58.345032 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b15eb2bb-1a9f-4b80-b297-48bdbf002e3e req-982b4375-d317-4672-92ec-c7380db120d9 service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4412122 May 24 10:02:58.400907 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-936459cb-5852-4cc1-8c99-54b9619719ea req-ca4f8c5d-9010-46fd-9a35-e858eb1c24a4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:58.402445 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-936459cb-5852-4cc1-8c99-54b9619719ea req-ca4f8c5d-9010-46fd-9a35-e858eb1c24a4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1627076 May 24 10:02:58.457150 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-936459cb-5852-4cc1-8c99-54b9619719ea req-fbaf3654-058f-448a-8462-ec8ce0c501d6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0465238 May 24 10:02:58.725164 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09a14880-8ac7-437e-974e-88228140bd8c req-7c131afe-7e5a-4963-8113-7fb280253be8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:58.727487 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-09a14880-8ac7-437e-974e-88228140bd8c req-7c131afe-7e5a-4963-8113-7fb280253be8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0997236 May 24 10:02:58.815733 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-09a14880-8ac7-437e-974e-88228140bd8c req-9768bdfe-7047-4787-a1c1-32e26e530a18 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0802009 May 24 10:02:58.957091 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e18e307-8a24-42f7-a188-fcfe115b2bd8 req-20a8e492-0d04-49e1-979e-4bc48130b8e6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:58.958642 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e18e307-8a24-42f7-a188-fcfe115b2bd8 req-20a8e492-0d04-49e1-979e-4bc48130b8e6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1023929 May 24 10:02:59.003716 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55884) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:02:59.024787 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2e18e307-8a24-42f7-a188-fcfe115b2bd8 req-700c28cb-68f6-4670-beed-e0073dbe08d4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0589805 May 24 10:02:59.268103 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-cd194f83-64d9-4ac5-9f89-cc5ba9282b01 service neutron] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.2620664 May 24 10:02:59.651898 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0234376-14c8-44ce-9235-db397202d265 req-8332c627-87bf-4ec7-8f30-35f7dbce194f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:59.653734 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0234376-14c8-44ce-9235-db397202d265 req-8332c627-87bf-4ec7-8f30-35f7dbce194f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 944 time: 0.1093023 May 24 10:02:59.709707 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0234376-14c8-44ce-9235-db397202d265 req-af04a436-4416-425a-bc29-b7bec234cf93 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0500140 May 24 10:02:59.734073 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-914a08ec-8261-4a25-ae9a-c52d235bd65a req-5352f3dd-042d-402b-8c86-52755cf130d6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:02:59.735680 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-914a08ec-8261-4a25-ae9a-c52d235bd65a req-5352f3dd-042d-402b-8c86-52755cf130d6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1204464 May 24 10:02:59.785343 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-914a08ec-8261-4a25-ae9a-c52d235bd65a req-e6b6b6ed-09b8-44ad-8c3e-1d27bf5f9ff6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431879 May 24 10:02:59.817832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 855e6647-516e-462d-8d07-0f5cf828da14 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:03:00.051681 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=855e6647-516e-462d-8d07-0f5cf828da14,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:00.116774 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '855e6647-516e-462d-8d07-0f5cf828da14', 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18', 'port_id': '855e6647-516e-462d-8d07-0f5cf828da14', 'mac_address': 'fa:16:3e:3e:cf:fc', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 63, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '9ee9ac65-ecfe-47d4-bbac-4561dabf43e6', 'ip_address': '172.20.0.14'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:03:00.145885 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0858dfc-0fd7-4acb-80b6-16cef3bc0b53 req-7c8ade3f-490d-4809-90ee-5c6b0bb53c50 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:00.147585 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d0858dfc-0fd7-4acb-80b6-16cef3bc0b53 req-7c8ade3f-490d-4809-90ee-5c6b0bb53c50 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1820066 May 24 10:03:00.266641 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d0858dfc-0fd7-4acb-80b6-16cef3bc0b53 req-91218b13-2551-44ed-8766-b9763422fd39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1126368 May 24 10:03:00.284775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:00.308796 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 855e6647-516e-462d-8d07-0f5cf828da14 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:03:00.338045 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=855e6647-516e-462d-8d07-0f5cf828da14,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:00.375116 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61c81420-1847-4223-ad1a-8edce6043151 req-87ffe249-b1e9-46b4-bba7-87315a23b4b9 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:00.377196 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-61c81420-1847-4223-ad1a-8edce6043151 req-87ffe249-b1e9-46b4-bba7-87315a23b4b9 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1480920 May 24 10:03:00.406103 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:00.408600 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:00.478903 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-61c81420-1847-4223-ad1a-8edce6043151 req-c249cb75-baa3-42f8-b24c-e049e5288058 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0922253 May 24 10:03:00.507440 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:00.514914 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:00.676658 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['855e6647-516e-462d-8d07-0f5cf828da14'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:03:01.076959 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c23ec2df-159b-429b-9a1a-14627dfffbf4 req-38361d6e-a832-458c-8208-79fc2a8b5d7b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.080984 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c23ec2df-159b-429b-9a1a-14627dfffbf4 req-38361d6e-a832-458c-8208-79fc2a8b5d7b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 945 time: 0.2048452 May 24 10:03:01.128653 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c23ec2df-159b-429b-9a1a-14627dfffbf4 req-1f23935f-3c4b-483e-957f-93e6e4e536eb tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0415707 May 24 10:03:01.329908 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3061944-d0ec-4ade-8f30-235293a24ee5 req-290ba513-fdab-47c1-8bbe-77f02aa63e4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.329908 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f3061944-d0ec-4ade-8f30-235293a24ee5 req-290ba513-fdab-47c1-8bbe-77f02aa63e4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1014783 May 24 10:03:01.393642 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f3061944-d0ec-4ade-8f30-235293a24ee5 req-c78af57c-e313-424a-ade7-898b8113c051 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0612350 May 24 10:03:01.477565 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2aa010f0-3039-42e6-a250-5a2e5ace9279 req-de17db34-1cbd-4f08-9c2c-e9212ae4244f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.484845 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2aa010f0-3039-42e6-a250-5a2e5ace9279 req-de17db34-1cbd-4f08-9c2c-e9212ae4244f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1961138 May 24 10:03:01.621740 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2aa010f0-3039-42e6-a250-5a2e5ace9279 req-3aa2daee-6bd7-40a0-adb8-3c47183399b5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1245329 May 24 10:03:01.721003 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f06d5fdd-7945-47eb-bda9-7e378fc308a1 req-d29ca57b-8fc4-4d88-88e2-3dee481de6c6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.722461 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f06d5fdd-7945-47eb-bda9-7e378fc308a1 req-d29ca57b-8fc4-4d88-88e2-3dee481de6c6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1760116 May 24 10:03:01.752096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49b75e31-feeb-42fb-9c98-7692080b650d req-f4f5c9c0-96c6-4103-9079-f91e4b66bbce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.754111 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-f4f5c9c0-96c6-4103-9079-f91e4b66bbce service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.1200626 May 24 10:03:01.817682 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 855e6647-516e-462d-8d07-0f5cf828da14 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:03:01.821281 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f06d5fdd-7945-47eb-bda9-7e378fc308a1 req-b033bbbb-6f5b-4e49-9da7-f4a20cd5c1d8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0919018 May 24 10:03:01.928954 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49b75e31-feeb-42fb-9c98-7692080b650d req-7ebd3b57-2a35-470e-91f9-87ddd0bd92c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:01.930487 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-7ebd3b57-2a35-470e-91f9-87ddd0bd92c9 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.1684704 May 24 10:03:01.988029 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 855e6647-516e-462d-8d07-0f5cf828da14 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:03:01.997844 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 855e6647-516e-462d-8d07-0f5cf828da14 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:03:01.997844 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.074151 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-099166fd-16c6-4df1-ba27-cf97345ee24a service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1000 time: 0.1370239 May 24 10:03:02.108238 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55956) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:02.194364 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49b75e31-feeb-42fb-9c98-7692080b650d req-bfa8477a-8f6d-4fdd-abcc-a1bc94a05c99 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:02.196035 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-bfa8477a-8f6d-4fdd-abcc-a1bc94a05c99 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.1084957 May 24 10:03:02.286397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.350608 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:03:02.358782 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49b75e31-feeb-42fb-9c98-7692080b650d req-e6e6363b-804d-4ec1-8684-f634a107b80b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:02.361965 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-e6e6363b-804d-4ec1-8684-f634a107b80b service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1580811 May 24 10:03:02.367269 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=855e6647-516e-462d-8d07-0f5cf828da14,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:02.369575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.421601 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:03:02.421809 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14', 'code': 200} May 24 10:03:02.422720 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Usage tracker for resource:security_group_rule and tenant:17ea1fb34cb84ae5b93921e285ad3580 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:03:02.427484 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:03:02.439004 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.440887 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:02.445755 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:03:02.458848 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97bfc3a0-b781-4dc3-bd57-4a589a404653 req-353027cf-abea-431f-9a69-7dd23fb14663 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:02.468028 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-97bfc3a0-b781-4dc3-bd57-4a589a404653 req-353027cf-abea-431f-9a69-7dd23fb14663 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 945 time: 0.1860671 May 24 10:03:02.538238 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-97bfc3a0-b781-4dc3-bd57-4a589a404653 req-924ad892-8c1e-46eb-b40d-31ee7ae87221 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0687871 May 24 10:03:02.613296 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:02.617175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:02.677263 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.769944 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:02.789082 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bc3e5092-291c-468f-9fac-d4707141f1fd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:02.847705 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62b3888f-be97-478e-ad8e-6d3aadc3f081 req-3f19a5f6-bff1-4615-bc41-34b577ac0a43 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:02.856808 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-62b3888f-be97-478e-ad8e-6d3aadc3f081 req-3f19a5f6-bff1-4615-bc41-34b577ac0a43 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.2329593 May 24 10:03:02.866616 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=17ea1fb34cb84ae5b93921e285ad3580&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7568774 May 24 10:03:02.869859 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 55976) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:02.872620 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e65a0333-7370-4cce-b697-c8ab0dfd761b,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:02.888899 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-e4655ebb-cd91-44fc-8940-460ab63ded17 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5205984 May 24 10:03:02.912578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:03.016857 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-62b3888f-be97-478e-ad8e-6d3aadc3f081 req-9a1b32da-c33c-4bcc-8805-7fbb1b190d1c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1589260 May 24 10:03:03.096570 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:03.127700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:03.170743 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49b75e31-feeb-42fb-9c98-7692080b650d req-e65867f3-db9f-4db1-80eb-7727ecdaecd5 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2556129 May 24 10:03:03.222215 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35e53396-b3b7-44fe-ba07-975eacbf7939 req-ac182465-e829-4442-8c30-31dbd9dac6d4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:03.224008 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-35e53396-b3b7-44fe-ba07-975eacbf7939 req-ac182465-e829-4442-8c30-31dbd9dac6d4 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 1099 time: 0.2738712 May 24 10:03:03.282296 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4db0a0db-c008-4265-8cff-9787fd9e6715 req-19b7d943-820e-4115-bb97-0eeb39faad64 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:03.289124 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4db0a0db-c008-4265-8cff-9787fd9e6715 req-19b7d943-820e-4115-bb97-0eeb39faad64 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.4199879 May 24 10:03:03.325228 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:03.338774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-352c1de2-20df-4dee-bd84-eda1f8a0c882 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:03.346403 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7f2b240e-034b-4e02-ac13-9db33d5264b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:03.371438 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-da9d37f9-53de-4b1f-b7b3-c7a0f419fac3 req-267db081-e47d-40ee-b0c1-3f6d8f963a96 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:03.373270 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-da9d37f9-53de-4b1f-b7b3-c7a0f419fac3 req-267db081-e47d-40ee-b0c1-3f6d8f963a96 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1221209 May 24 10:03:03.409401 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4db0a0db-c008-4265-8cff-9787fd9e6715 req-cce29ff8-4083-42be-a473-482be021c8ad tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1192896 May 24 10:03:03.438418 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e65a0333-7370-4cce-b697-c8ab0dfd761b,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:03.442919 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:03.451848 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-35e53396-b3b7-44fe-ba07-975eacbf7939 req-c2d442fb-dea3-4765-9d5d-6d1049552e85 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2142031 May 24 10:03:03.455332 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:03.457798 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-cdf1eac3-a473-4968-8a85-b4f738a10a5c admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e65a0333-7370-4cce-b697-c8ab0dfd761b HTTP/1.1" status: 204 len: 168 time: 0.5860674 May 24 10:03:03.460626 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-da9d37f9-53de-4b1f-b7b3-c7a0f419fac3 req-b203c420-3473-4e64-9ebd-3304333217b4 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0781279 May 24 10:03:03.807995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55d621cb-7748-4526-966a-a1c2ba8f76d5 req-b3208b63-c2ea-4702-b0fd-b6edfae29294 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:03.815365 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-55d621cb-7748-4526-966a-a1c2ba8f76d5 req-b3208b63-c2ea-4702-b0fd-b6edfae29294 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.1540198 May 24 10:03:03.865851 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7939995c-30b0-4e6f-8728-642df43d147f req-a77d5dcb-e6c5-456b-90db-68062636d77a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:03.867501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-a77d5dcb-e6c5-456b-90db-68062636d77a service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 1097 time: 0.1604960 May 24 10:03:03.870382 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-55d621cb-7748-4526-966a-a1c2ba8f76d5 req-8952e8ee-3fef-4d70-a79f-27698d2031e3 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0541191 May 24 10:03:04.028971 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7939995c-30b0-4e6f-8728-642df43d147f req-e4202b30-7ef0-4019-bc5b-5e2b07cd0693 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.031361 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-e4202b30-7ef0-4019-bc5b-5e2b07cd0693 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1097 time: 0.1255095 May 24 10:03:04.239057 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a4be26b-8ab8-48d8-aa91-2a72142888be req-6ad41967-3873-462a-b3b9-a251b39df5b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.247275 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5a4be26b-8ab8-48d8-aa91-2a72142888be req-6ad41967-3873-462a-b3b9-a251b39df5b6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0912209 May 24 10:03:04.253317 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7939995c-30b0-4e6f-8728-642df43d147f req-60f7e210-ec5c-48f1-bfcc-3cd38251b803 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.254967 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-60f7e210-ec5c-48f1-bfcc-3cd38251b803 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.2160518 May 24 10:03:04.309533 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5a4be26b-8ab8-48d8-aa91-2a72142888be req-d6ca19a9-d2c7-419d-9680-2019ab654be0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0593147 May 24 10:03:04.402538 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-b62f8894-8496-4e1a-895b-326197e63f45 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 999 time: 0.1395657 May 24 10:03:04.499891 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7939995c-30b0-4e6f-8728-642df43d147f req-0815f1f6-749e-4b84-a57f-fd464846d213 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.501763 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-0815f1f6-749e-4b84-a57f-fd464846d213 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0885487 May 24 10:03:04.614724 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7939995c-30b0-4e6f-8728-642df43d147f req-f9b1b5f3-9fab-405a-8107-640cd10ad3cf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.616372 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-f9b1b5f3-9fab-405a-8107-640cd10ad3cf service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1055787 May 24 10:03:04.750469 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4cb96da-16fa-4812-a6b4-132e67dac8d6 req-c41c9183-aad6-4870-b252-bf1e766ed999 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.762554 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b4cb96da-16fa-4812-a6b4-132e67dac8d6 req-c41c9183-aad6-4870-b252-bf1e766ed999 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1191423 May 24 10:03:04.822557 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b4cb96da-16fa-4812-a6b4-132e67dac8d6 req-4e370e40-d844-47c9-8537-79d5061e7750 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0587251 May 24 10:03:04.829128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cce2f8a7-86f1-44b5-997a-5bb2555f0d05 req-b722c6e0-ef74-4993-a120-0f5b819198b4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:04.830658 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cce2f8a7-86f1-44b5-997a-5bb2555f0d05 req-b722c6e0-ef74-4993-a120-0f5b819198b4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2129757 May 24 10:03:04.832724 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-61a2e21d-72a7-41ba-89ee-9689c97c2bf8 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1756401 May 24 10:03:05.068818 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cce2f8a7-86f1-44b5-997a-5bb2555f0d05 req-f3c3024b-d42a-4f8c-9ae9-3b2d4fe75a76 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2284112 May 24 10:03:05.117641 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-8ba8cad3-5cce-4a09-bf00-5740eff5ab3d service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2753985 May 24 10:03:05.151048 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 56018) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:05.209184 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4acb20be-06cc-4f11-9cd9-426ed2f6c124 req-95925890-bb1c-4869-b2bd-01ef65f8688b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:05.211349 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4acb20be-06cc-4f11-9cd9-426ed2f6c124 req-95925890-bb1c-4869-b2bd-01ef65f8688b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.1700897 May 24 10:03:05.312939 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4acb20be-06cc-4f11-9cd9-426ed2f6c124 req-63a759c4-6f15-48e6-bd43-eeeb1e7900db tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0948522 May 24 10:03:05.409704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.437016 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:03:05.441249 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Usage tracker for resource:security_group_rule and tenant:6e633ff6f0534e578def9d778810e57e is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:03:05.444752 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:03:05.452030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:03:05.541219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9cdb65ef-afeb-48fb-ba40-8c9dc5b2a6ac req-1941ebe9-3606-41ef-8bbf-8ae77b1d87e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:05.542993 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9cdb65ef-afeb-48fb-ba40-8c9dc5b2a6ac req-1941ebe9-3606-41ef-8bbf-8ae77b1d87e5 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 1098 time: 0.1559904 May 24 10:03:05.602092 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.646600 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9cdb65ef-afeb-48fb-ba40-8c9dc5b2a6ac req-843e6476-bc4b-410c-9223-411264b8c9f8 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0483434 May 24 10:03:05.690746 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71ce482d-1273-4d10-9499-c6f01246a184 req-61fcb527-2aae-43d2-a454-ea805c63da5b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:05.701556 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-71ce482d-1273-4d10-9499-c6f01246a184 req-61fcb527-2aae-43d2-a454-ea805c63da5b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1547589 May 24 10:03:05.723927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.764512 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5792df75-87b3-4b1c-8788-46e88c1d2dda admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:05.765403 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-71ce482d-1273-4d10-9499-c6f01246a184 req-493977cc-66ac-4cd8-8c9e-ca6eb034adc5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0600538 May 24 10:03:05.786809 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6e633ff6f0534e578def9d778810e57e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6346834 May 24 10:03:05.789774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56054) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:05.810508 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-17e9cac7-538d-412c-afe3-57a3ce0cde1a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.811083 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=aa080276-004e-4754-a37c-45f641456223,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:05.830145 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:05.833635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.919032 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:05.964499 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:06.088367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d5a993b-0230-428b-9a59-b790e3d72d33 req-ae09b400-5373-4e23-bc19-263e8a9cf826 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:06.089903 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d5a993b-0230-428b-9a59-b790e3d72d33 req-ae09b400-5373-4e23-bc19-263e8a9cf826 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0899787 May 24 10:03:06.112376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:06.121295 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-00da7703-fa20-4f3e-9edf-a569d5a05c57 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:06.152130 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:06.152130 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/aa080276-004e-4754-a37c-45f641456223 HTTP/1.1" status: 204 len: 168 time: 0.3441567 May 24 10:03:06.152130 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=aa080276-004e-4754-a37c-45f641456223,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:06.152130 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:06.157372 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9d5a993b-0230-428b-9a59-b790e3d72d33 req-0361031e-3f7d-4991-9443-99aafce8d47c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0612729 May 24 10:03:07.078642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c581cc51-fa92-4e8f-9aa9-71d97d44a281 req-25bc29e7-f5a1-4ed3-822e-471fcc07197c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:07.089555 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c581cc51-fa92-4e8f-9aa9-71d97d44a281 req-25bc29e7-f5a1-4ed3-822e-471fcc07197c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1259739 May 24 10:03:07.175774 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:07.207030 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c581cc51-fa92-4e8f-9aa9-71d97d44a281 req-f5949d97-0898-4e04-ba1f-d510a29607e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1161585 May 24 10:03:07.453421 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71f0719a-49b2-44ca-88af-0278f23685ed req-bd925215-5542-41a5-a158-992b96c07ce1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:07.465465 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-71f0719a-49b2-44ca-88af-0278f23685ed req-bd925215-5542-41a5-a158-992b96c07ce1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.1432383 May 24 10:03:07.591755 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-71f0719a-49b2-44ca-88af-0278f23685ed req-2ca56ead-ad94-4371-b862-1bbd2db2ec71 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1250093 May 24 10:03:07.908537 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8a79401-01ca-46f3-8eaa-a6408437540d req-ffc50f45-4e39-4576-8989-21288bcccf46 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:07.910544 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c8a79401-01ca-46f3-8eaa-a6408437540d req-ffc50f45-4e39-4576-8989-21288bcccf46 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1147702 May 24 10:03:08.085304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c8a79401-01ca-46f3-8eaa-a6408437540d req-fa436454-a9d4-4cb3-aa8f-c448adf3c9b6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1646800 May 24 10:03:08.591287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4714b29-2685-4949-8223-f29ad29bb874 req-3016e9bf-58d4-4b4b-95ba-9a38e391736b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:08.597344 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b4714b29-2685-4949-8223-f29ad29bb874 req-3016e9bf-58d4-4b4b-95ba-9a38e391736b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1833239 May 24 10:03:08.676840 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b4714b29-2685-4949-8223-f29ad29bb874 req-f528c23b-e6f4-4bad-bf2b-b0e70f1b0862 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0743804 May 24 10:03:08.753740 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:10.171062 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40b23fff-f824-4e5a-9720-d641ccbc6788 req-d7188e26-ef1b-429f-a0e3-a71162e0fc51 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:10.172671 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-40b23fff-f824-4e5a-9720-d641ccbc6788 req-d7188e26-ef1b-429f-a0e3-a71162e0fc51 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0870979 May 24 10:03:10.228182 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-40b23fff-f824-4e5a-9720-d641ccbc6788 req-7a8625a2-8bf7-4580-8753-1e9260c79585 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0492697 May 24 10:03:11.461600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-627f8647-4401-4729-822a-4d1d583536a6 req-59e35657-eb53-4d30-91a7-093e7c0f8d71 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:11.463692 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-627f8647-4401-4729-822a-4d1d583536a6 req-59e35657-eb53-4d30-91a7-093e7c0f8d71 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0996680 May 24 10:03:11.517947 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-627f8647-4401-4729-822a-4d1d583536a6 req-58b817f8-4563-4ae5-86e6-f17cce255a39 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0482414 May 24 10:03:11.817206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c199fd4b-59fe-41ce-b284-4a0ae1535541 req-4a09b48d-609a-47aa-a785-4158bf259e1d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:11.818611 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c199fd4b-59fe-41ce-b284-4a0ae1535541 req-4a09b48d-609a-47aa-a785-4158bf259e1d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0844433 May 24 10:03:11.869073 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c199fd4b-59fe-41ce-b284-4a0ae1535541 req-53cd2db7-b95c-4427-9295-a4b63b00b786 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0438056 May 24 10:03:12.448135 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:03:12.760799 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f84e63c5-2ec7-4a20-9999-7698eee989b0 req-0ec8134f-8193-4d66-8668-b5e8325ae02e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:12.762275 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f84e63c5-2ec7-4a20-9999-7698eee989b0 req-0ec8134f-8193-4d66-8668-b5e8325ae02e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0924320 May 24 10:03:12.819995 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f84e63c5-2ec7-4a20-9999-7698eee989b0 req-da962b6c-0d36-4cb8-97a4-a6f8fd9dcecf tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0513389 May 24 10:03:13.079210 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc70d2ff-01de-4fd2-8e73-ee8ab2ce7585 req-2077ede5-442b-4db8-8da9-918232a45b37 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:13.085731 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fc70d2ff-01de-4fd2-8e73-ee8ab2ce7585 req-2077ede5-442b-4db8-8da9-918232a45b37 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1105697 May 24 10:03:13.138486 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fc70d2ff-01de-4fd2-8e73-ee8ab2ce7585 req-11d01676-5a26-4ef9-bd44-80d990385e22 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0517495 May 24 10:03:14.025278 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26e715d7-e205-43d3-a787-0b6ced215c80 req-ba47521c-14fa-4b1e-9eea-46b5e1010237 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:14.033321 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-26e715d7-e205-43d3-a787-0b6ced215c80 req-ba47521c-14fa-4b1e-9eea-46b5e1010237 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0906415 May 24 10:03:14.077354 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-26e715d7-e205-43d3-a787-0b6ced215c80 req-f8936100-e97f-467f-81d9-268e7ddf3407 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0427675 May 24 10:03:14.361218 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1977db18-03b3-41b4-a4af-91919f1052bc req-7212bc96-7a42-4c13-aa5d-230d931db0aa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:14.362757 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1977db18-03b3-41b4-a4af-91919f1052bc req-7212bc96-7a42-4c13-aa5d-230d931db0aa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0954955 May 24 10:03:14.427747 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1977db18-03b3-41b4-a4af-91919f1052bc req-63eef314-6b31-4994-b698-55b4e442d107 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0499160 May 24 10:03:15.279082 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4410e35a-e3d9-4b90-aadd-6630a1e74302 req-348a0e76-1e14-4acf-8541-21788e29e394 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:15.282510 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4410e35a-e3d9-4b90-aadd-6630a1e74302 req-348a0e76-1e14-4acf-8541-21788e29e394 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0891347 May 24 10:03:15.331756 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4410e35a-e3d9-4b90-aadd-6630a1e74302 req-b308e2e4-c3db-42ee-a309-6411d8352de6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0449495 May 24 10:03:15.641634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47a6ac3f-ec74-43cf-91ab-a859aeddb123 req-14dbcfc1-c946-46d9-b8e6-61540648dcd3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:15.643170 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47a6ac3f-ec74-43cf-91ab-a859aeddb123 req-14dbcfc1-c946-46d9-b8e6-61540648dcd3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0886426 May 24 10:03:15.690140 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47a6ac3f-ec74-43cf-91ab-a859aeddb123 req-380857ab-f86e-4ce1-81a4-a72efaba50a2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410731 May 24 10:03:16.538149 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4037e408-9f50-4eef-8bc0-98950df307c1 req-6aff9cf1-01fb-4308-aa68-7de2b7bfec2b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:16.539547 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4037e408-9f50-4eef-8bc0-98950df307c1 req-6aff9cf1-01fb-4308-aa68-7de2b7bfec2b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0872877 May 24 10:03:16.584837 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4037e408-9f50-4eef-8bc0-98950df307c1 req-2e8cf7a8-0f31-4377-b805-a68cedb23498 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0388043 May 24 10:03:16.891225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-836ff58f-9e2c-4e2d-9045-684c0de89582 req-fbfbc707-7138-4bf4-8adc-01185916cbc6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:16.892625 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-836ff58f-9e2c-4e2d-9045-684c0de89582 req-fbfbc707-7138-4bf4-8adc-01185916cbc6 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0909188 May 24 10:03:16.938433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-836ff58f-9e2c-4e2d-9045-684c0de89582 req-95cdecbd-4d29-4093-8d87-fc469c2dd7b2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401368 May 24 10:03:17.792842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-958ae011-5091-428e-ab36-e5d093460609 req-f306eb56-6242-458a-a33d-1f53c3288a3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:17.794359 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-958ae011-5091-428e-ab36-e5d093460609 req-f306eb56-6242-458a-a33d-1f53c3288a3f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0893121 May 24 10:03:17.839086 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-958ae011-5091-428e-ab36-e5d093460609 req-cf804ae8-d6b5-4744-b848-dee56bc5511c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390162 May 24 10:03:18.174845 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6c9beff-5991-4014-a465-c649d9e98207 req-4d287ac7-8cfd-46f3-ae9a-fce3d60e7f9e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:18.176400 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6c9beff-5991-4014-a465-c649d9e98207 req-4d287ac7-8cfd-46f3-ae9a-fce3d60e7f9e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1168907 May 24 10:03:18.179141 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-c9e89ce3-a980-496b-9f8c-549e7f584bd7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:18.180544 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-c9e89ce3-a980-496b-9f8c-549e7f584bd7 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.1524477 May 24 10:03:18.307085 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6c9beff-5991-4014-a465-c649d9e98207 req-220635f7-9fbe-4622-a507-0efccbdcd428 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1243641 May 24 10:03:18.352013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-50756dab-f967-4dce-8a9d-c4a9c75e16f4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:18.353650 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-50756dab-f967-4dce-8a9d-c4a9c75e16f4 service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.1561475 May 24 10:03:18.524198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-6370cd6d-9c88-4139-bf0e-2b3bff36a73c service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 217 time: 0.1506772 May 24 10:03:18.574305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-2e36f828-f2d2-4e31-aefc-bf5c8209f1ba service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:18.575736 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-2e36f828-f2d2-4e31-aefc-bf5c8209f1ba service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.0585017 May 24 10:03:18.625554 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-200bb054-f748-46ca-8026-10c95d40e364 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0413184 May 24 10:03:18.768831 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-96454a1f-0ae4-407a-8cc4-6dae2eaf4556 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1282568 May 24 10:03:18.914746 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a4715b47-5eab-4392-ade0-e79d828020d8 req-9a301f06-dc19-498e-9ca9-09987d149269 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1377842 May 24 10:03:19.038996 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ce2cf4-c866-4140-9231-050a5ea75581 req-e298202a-0afd-44d9-812e-d6d8c00e206f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:19.041087 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-88ce2cf4-c866-4140-9231-050a5ea75581 req-e298202a-0afd-44d9-812e-d6d8c00e206f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0824087 May 24 10:03:19.087861 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-88ce2cf4-c866-4140-9231-050a5ea75581 req-dee5abee-8736-4d04-aa4d-982a19a72e9c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0402570 May 24 10:03:19.516792 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e20b07fe-2a2e-4c5a-bb05-9680ad707ccc req-72796bf2-956f-4956-8ce1-473098d4db36 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:19.518471 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e20b07fe-2a2e-4c5a-bb05-9680ad707ccc req-72796bf2-956f-4956-8ce1-473098d4db36 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0957220 May 24 10:03:19.566654 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e20b07fe-2a2e-4c5a-bb05-9680ad707ccc req-e0243866-87ef-4066-a4b4-7041d18d31ec tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414174 May 24 10:03:20.111872 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:20.344395 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0da2abe5-b882-4313-acd0-74b6a2ec1157 req-88cbbe60-a37c-4b30-a35a-82188d87b022 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:20.345906 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0da2abe5-b882-4313-acd0-74b6a2ec1157 req-88cbbe60-a37c-4b30-a35a-82188d87b022 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.1462722 May 24 10:03:20.391779 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0da2abe5-b882-4313-acd0-74b6a2ec1157 req-91468e7d-cafc-472c-a1a2-824f46c883f3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0398512 May 24 10:03:20.752936 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6380360c-1edd-4264-b0fd-ddc094a4346c req-58593ab5-42a8-460f-927f-9884ebc85710 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:20.754433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6380360c-1edd-4264-b0fd-ddc094a4346c req-58593ab5-42a8-460f-927f-9884ebc85710 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0807595 May 24 10:03:20.802623 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6380360c-1edd-4264-b0fd-ddc094a4346c req-0ecb31d4-d045-42ba-b43b-0d1b51575093 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418816 May 24 10:03:21.021750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56272) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:21.085466 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Deleting subnet 1f8e051e-bd56-4915-8e94-622f6ff108d3 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:03:21.088366 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.138442 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.203369 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.599624 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04b6b740-abdf-4ee8-8a18-1924fa86a920 req-9af40f8d-5b3f-47fe-a76c-bb1917ac657e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:21.601108 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-04b6b740-abdf-4ee8-8a18-1924fa86a920 req-9af40f8d-5b3f-47fe-a76c-bb1917ac657e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0915928 May 24 10:03:21.646566 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-04b6b740-abdf-4ee8-8a18-1924fa86a920 req-bfee8176-64cb-45b7-acc9-dfb98607abc8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392592 May 24 10:03:21.660876 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.666140 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:21.704661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1f8e051e-bd56-4915-8e94-622f6ff108d3,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:21.707710 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:21.708689 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.715363 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-81a544a4-866b-44d8-9235-b82c9ed24637 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/1f8e051e-bd56-4915-8e94-622f6ff108d3 HTTP/1.1" status: 204 len: 168 time: 0.6925516 May 24 10:03:21.717924 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56280) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:21.830304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:21.989617 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.016508 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['570c27de-a372-4133-bc81-83e9d6aa9b15']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:03:22.031933 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '0af48d33-b0f2-492b-b676-993923c0f33a') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:03:22.032225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '0af48d33-b0f2-492b-b676-993923c0f33a') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:03:22.048212 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.070194 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.079107 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Releasing vlan 82 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:03:22.079485 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:03:22.092679 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:03:22.157657 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5deb7f11-d78e-4071-b532-f4772f607724 req-d3a54bd4-4b93-49b8-8036-e0d81d903ab3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:22.159107 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5deb7f11-d78e-4071-b532-f4772f607724 req-d3a54bd4-4b93-49b8-8036-e0d81d903ab3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.2061303 May 24 10:03:22.212579 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5deb7f11-d78e-4071-b532-f4772f607724 req-21d7f4b8-1496-479b-ae43-3b6fa669ef75 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0477662 May 24 10:03:22.272020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.276502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-77ed13d8-117c-4706-af1a-21dd73756013 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:22.369976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:22.380541 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Pushing event deleted for resources: {'Network': ['ID=570c27de-a372-4133-bc81-83e9d6aa9b15,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:22.384615 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:22.461136 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-973120b1-d02d-4fe0-b584-587a6d275580 admin admin] 10.222.0.67 "DELETE /v2.0/networks/570c27de-a372-4133-bc81-83e9d6aa9b15 HTTP/1.1" status: 204 len: 168 time: 0.7421567 May 24 10:03:22.847438 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39c30fd6-29e6-41ea-b311-0e97aca01ec4 req-9a59ac10-23d3-4ce4-974a-279cdd51966d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:22.848911 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-39c30fd6-29e6-41ea-b311-0e97aca01ec4 req-9a59ac10-23d3-4ce4-974a-279cdd51966d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0867817 May 24 10:03:22.897283 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-39c30fd6-29e6-41ea-b311-0e97aca01ec4 req-9938bf9c-d0ea-4dc0-9835-1a6f9163d2d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0420539 May 24 10:03:22.912770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56292) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:22.967093 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-b2aa5eec-5f45-4e2b-87f4-cd1c85975f09 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6f38795dc42f4d5186770d3b222547ae&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0531158 May 24 10:03:22.969597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56296) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:03:23.027670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:23.054701 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:23.163646 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:23.168216 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0530b525-6ee5-4602-b8c9-0e3c9239a303 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:23.184467 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d467c95d-fd2f-4e55-82e5-b5b76f6881e7,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:23.187616 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:23.265077 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:23.266256 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/d467c95d-fd2f-4e55-82e5-b5b76f6881e7 HTTP/1.1" status: 204 len: 168 time: 0.2957611 May 24 10:03:23.414163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebf6d474-fe9f-4631-8d8d-69769f77aeb9 req-d52d1dd0-02d4-43f5-ad7d-902b03b639b3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:23.415590 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ebf6d474-fe9f-4631-8d8d-69769f77aeb9 req-d52d1dd0-02d4-43f5-ad7d-902b03b639b3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0858371 May 24 10:03:23.461918 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ebf6d474-fe9f-4631-8d8d-69769f77aeb9 req-77ad985d-9ee0-4f4d-980b-e71d3b37d849 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394952 May 24 10:03:24.096339 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3387ac9a-eb04-439b-b9ab-f3d570ac71f2 req-7157e94a-3ef3-4c6e-825a-e08f9e95c3af tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:24.097835 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3387ac9a-eb04-439b-b9ab-f3d570ac71f2 req-7157e94a-3ef3-4c6e-825a-e08f9e95c3af tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0883110 May 24 10:03:24.143375 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3387ac9a-eb04-439b-b9ab-f3d570ac71f2 req-5a056381-626f-481f-a684-145525e2f258 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392029 May 24 10:03:24.652630 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3cbe9873-e99f-4984-b54d-1dd2847c624b req-37fb2aac-6c1c-4b56-a10e-13e1dcf4eb08 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:24.654350 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3cbe9873-e99f-4984-b54d-1dd2847c624b req-37fb2aac-6c1c-4b56-a10e-13e1dcf4eb08 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0859451 May 24 10:03:24.703236 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3cbe9873-e99f-4984-b54d-1dd2847c624b req-5414450d-8ad4-4120-b52a-eb1f1832c7b0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422726 May 24 10:03:25.345893 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2330c67-a134-41c7-87c9-487e764b06d6 req-4dc9fccf-bca1-48eb-aa52-f72d5de6e4ab tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:25.347400 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2330c67-a134-41c7-87c9-487e764b06d6 req-4dc9fccf-bca1-48eb-aa52-f72d5de6e4ab tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0805187 May 24 10:03:25.393133 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2330c67-a134-41c7-87c9-487e764b06d6 req-ca5adceb-36c0-40f2-9d79-0e1a62569f1b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391092 May 24 10:03:25.905336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-258fa011-3dc9-43af-8c7a-cb651f8bb484 req-d34a78f1-4c5c-4cb5-b4a9-d3fe911d8688 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:25.906988 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-258fa011-3dc9-43af-8c7a-cb651f8bb484 req-d34a78f1-4c5c-4cb5-b4a9-d3fe911d8688 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0876372 May 24 10:03:25.956957 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-258fa011-3dc9-43af-8c7a-cb651f8bb484 req-8392431e-6155-48b2-bd98-c6d986208863 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400510 May 24 10:03:26.592190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e35be8b-d7f2-4fdd-b102-e3dcbc699988 req-61ff87d3-0301-4d50-a915-4e352f02a654 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:26.594106 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1e35be8b-d7f2-4fdd-b102-e3dcbc699988 req-61ff87d3-0301-4d50-a915-4e352f02a654 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0865588 May 24 10:03:26.639705 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1e35be8b-d7f2-4fdd-b102-e3dcbc699988 req-24cfc6de-68d8-49ef-b41f-d99b4c58c576 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0396016 May 24 10:03:27.152827 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00ac585a-f758-48fc-b800-efa52dc6d522 req-93c15158-fe07-4d60-a7bd-b288ad51c8aa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:27.154369 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-00ac585a-f758-48fc-b800-efa52dc6d522 req-93c15158-fe07-4d60-a7bd-b288ad51c8aa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0879493 May 24 10:03:27.201094 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-00ac585a-f758-48fc-b800-efa52dc6d522 req-8bb60def-fa58-4f04-938e-2a5d8d657c89 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402727 May 24 10:03:27.834935 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fc6af24-eca5-4282-b163-08f35eee8dd7 req-7c0e545d-2929-4a43-be40-0d753e99a3ed tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:27.836427 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4fc6af24-eca5-4282-b163-08f35eee8dd7 req-7c0e545d-2929-4a43-be40-0d753e99a3ed tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 944 time: 0.0839407 May 24 10:03:27.897504 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4fc6af24-eca5-4282-b163-08f35eee8dd7 req-20c8c8f3-2972-42ff-b1e1-951340f38f4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0553720 May 24 10:03:28.007114 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:03:28.143691 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:28.160757 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'mac_address': 'fa:16:3e:ae:4f:d7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:03:28.216122 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:28.234314 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:03:28.251037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:28.346624 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:28.348572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:28.394133 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77145a80-99ef-46ed-be77-819584728e40 req-8af2bafa-05aa-41b5-8e5e-ea78a0bb7179 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:28.395603 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-77145a80-99ef-46ed-be77-819584728e40 req-8af2bafa-05aa-41b5-8e5e-ea78a0bb7179 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0847058 May 24 10:03:28.415807 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=9']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:28.420068 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:28.446691 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-77145a80-99ef-46ed-be77-819584728e40 req-e374f83f-eac1-4c4c-bc26-ae591404497a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448585 May 24 10:03:28.676037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['6fec3696-3d1d-4762-9674-fe87fd37e211'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:03:29.090784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-301cf58a-5f5b-46a2-a0a5-e8502a40d41b req-96a9ead4-1fc3-48d4-b027-2c9b7010c648 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:29.092430 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-301cf58a-5f5b-46a2-a0a5-e8502a40d41b req-96a9ead4-1fc3-48d4-b027-2c9b7010c648 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 945 time: 0.0802736 May 24 10:03:29.138749 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-301cf58a-5f5b-46a2-a0a5-e8502a40d41b req-47c3e32c-663b-46b3-a9ed-8c490aec0056 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0403597 May 24 10:03:29.635850 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e13b2f39-0fa0-471b-98df-fd0fa6ec13b0 req-d932ea8b-1437-47ee-9080-40d835d73514 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:29.637297 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e13b2f39-0fa0-471b-98df-fd0fa6ec13b0 req-d932ea8b-1437-47ee-9080-40d835d73514 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0808983 May 24 10:03:29.684584 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e13b2f39-0fa0-471b-98df-fd0fa6ec13b0 req-5d2d2484-8ecb-4f64-8fbb-ac7aeb5b72fa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404038 May 24 10:03:30.028494 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:03:30.098532 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 6fec3696-3d1d-4762-9674-fe87fd37e211 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:03:30.098913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:30.215918 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:30.249606 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:03:30.260657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:03:30.309384 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'code': 200} May 24 10:03:30.329183 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:30.331239 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56671eb1-9357-43ea-8198-99f394312b4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:03:30.343539 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f857616-0562-4f39-92e8-c24fe20c9ed8 req-ea89d836-2c5b-4037-84a0-8adafcaeb50b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:30.345294 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2f857616-0562-4f39-92e8-c24fe20c9ed8 req-ea89d836-2c5b-4037-84a0-8adafcaeb50b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 945 time: 0.0940759 May 24 10:03:30.391442 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2f857616-0562-4f39-92e8-c24fe20c9ed8 req-e25a2429-2460-45d5-9a76-300093b38335 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0396786 May 24 10:03:30.398510 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=10']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:03:30.402220 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:03:30.883104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a208510-fb50-4a7e-8e66-c05782bc5ee0 req-d4feb381-2615-4374-90d7-866fab927f7e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:30.884634 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4a208510-fb50-4a7e-8e66-c05782bc5ee0 req-d4feb381-2615-4374-90d7-866fab927f7e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0891528 May 24 10:03:30.929675 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4a208510-fb50-4a7e-8e66-c05782bc5ee0 req-d3d6b11f-f237-4e62-a08b-d4681c1aad69 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384681 May 24 10:03:31.092299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9ce1164-4baf-4d1e-9e01-51a2368d5666 req-c4469b3d-2a9f-4c7c-a4ff-f3f16738b03c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:31.093832 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c9ce1164-4baf-4d1e-9e01-51a2368d5666 req-c4469b3d-2a9f-4c7c-a4ff-f3f16738b03c service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1100 time: 0.0706594 May 24 10:03:31.139429 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c9ce1164-4baf-4d1e-9e01-51a2368d5666 req-1c394017-7dcf-40e7-9a30-7b73d97ac3e0 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0363348 May 24 10:03:31.584260 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-090db1b6-b427-4d6c-a2d6-8d18ec4ab327 req-6e6c1679-a1ef-4900-b032-a61d6d3e2373 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:31.585758 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-090db1b6-b427-4d6c-a2d6-8d18ec4ab327 req-6e6c1679-a1ef-4900-b032-a61d6d3e2373 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0801930 May 24 10:03:31.631674 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-090db1b6-b427-4d6c-a2d6-8d18ec4ab327 req-a05e2671-71fc-4c76-a8d3-63aa51567425 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0395744 May 24 10:03:32.119054 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-347942f9-0f0a-45d0-9067-b60a4a8b12cf req-fe76e6a4-ad03-4a4a-bebf-2a316357102e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:32.121122 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-347942f9-0f0a-45d0-9067-b60a4a8b12cf req-fe76e6a4-ad03-4a4a-bebf-2a316357102e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0843282 May 24 10:03:32.166037 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-347942f9-0f0a-45d0-9067-b60a4a8b12cf req-bc94d0e2-9265-4526-8b81-083d500cdeae tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392635 May 24 10:03:32.834992 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb294feb-66b1-4b23-afc7-c46cb27f863d req-aa073ed9-9364-4891-a151-11c81788a777 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:32.836460 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-eb294feb-66b1-4b23-afc7-c46cb27f863d req-aa073ed9-9364-4891-a151-11c81788a777 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0886505 May 24 10:03:32.885897 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-eb294feb-66b1-4b23-afc7-c46cb27f863d req-e50a3f2d-37bc-4b7a-91be-6614d1200fdb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433276 May 24 10:03:33.288407 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9ed8c2aa-fd27-4a0e-a7f0-2e14230f7e68 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:33.359016 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-54f32628-8ec7-44b9-82e1-ad989e0f8976 req-8503bd0c-27f4-4a35-b286-0e8f14499a63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:33.360536 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-54f32628-8ec7-44b9-82e1-ad989e0f8976 req-8503bd0c-27f4-4a35-b286-0e8f14499a63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0862536 May 24 10:03:33.407342 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-54f32628-8ec7-44b9-82e1-ad989e0f8976 req-1f74ecc2-6cc9-460d-b18b-ce232bfcff47 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0411935 May 24 10:03:34.092320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9b31844-0b0b-4c99-bf7e-e513375e99d6 req-8fc25ac8-d79c-4865-9d6e-b0d895ed0279 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:34.093895 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9b31844-0b0b-4c99-bf7e-e513375e99d6 req-8fc25ac8-d79c-4865-9d6e-b0d895ed0279 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0844042 May 24 10:03:34.143586 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9b31844-0b0b-4c99-bf7e-e513375e99d6 req-fd67ab69-a9e5-4b83-ab59-4ae52b309a2b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0438938 May 24 10:03:34.610737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ff37ed6-e260-4b20-a0a2-fe1e9d392d5b req-dd850d4c-1e8a-45b9-98d2-33d78455441c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:34.614974 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1ff37ed6-e260-4b20-a0a2-fe1e9d392d5b req-dd850d4c-1e8a-45b9-98d2-33d78455441c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0972385 May 24 10:03:34.664464 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1ff37ed6-e260-4b20-a0a2-fe1e9d392d5b req-7553f62e-0986-4551-a6b6-fe86fd22c531 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436571 May 24 10:03:35.336769 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64627887-c499-4552-a7e0-cf922fed9257 req-f8a4e7ab-02b6-4116-85da-624799ca0ac6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:35.338266 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-64627887-c499-4552-a7e0-cf922fed9257 req-f8a4e7ab-02b6-4116-85da-624799ca0ac6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0830200 May 24 10:03:35.390663 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-64627887-c499-4552-a7e0-cf922fed9257 req-7dcc31db-7033-444f-b313-8ea8a8a13c60 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0462995 May 24 10:03:35.856848 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-420ac11f-4015-4194-b7ea-9812c3f34e9f req-7c4f3041-5b2c-45cf-b7b3-d7272a4471d9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:35.858371 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-420ac11f-4015-4194-b7ea-9812c3f34e9f req-7c4f3041-5b2c-45cf-b7b3-d7272a4471d9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0866923 May 24 10:03:35.865849 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e2b19bf5-b823-43c8-a700-6c907385d25e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:35.894739 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:35.907420 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-420ac11f-4015-4194-b7ea-9812c3f34e9f req-a8839296-0a10-4974-b785-23af0a169a34 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430336 May 24 10:03:36.311528 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-565a30fd-3531-45e1-a937-6758b73b7908 req-fde8a509-36ed-4373-b394-e969eeb06b9c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:36.312981 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-565a30fd-3531-45e1-a937-6758b73b7908 req-fde8a509-36ed-4373-b394-e969eeb06b9c service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.0745182 May 24 10:03:36.362557 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-565a30fd-3531-45e1-a937-6758b73b7908 req-3aa7ca87-b25c-4fee-b989-8c9085949de9 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379021 May 24 10:03:37.105503 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3809846-2dc7-4e7d-b344-08d52066613e req-ff7414c2-45ce-44d7-847e-a5425d85e6f0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:37.106967 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c3809846-2dc7-4e7d-b344-08d52066613e req-ff7414c2-45ce-44d7-847e-a5425d85e6f0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0835996 May 24 10:03:37.150935 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c3809846-2dc7-4e7d-b344-08d52066613e req-04aed662-08f3-491c-9fb9-f92f268174ee tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381742 May 24 10:03:37.174775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:38.343139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c51bb62f-04df-45f5-803b-f1d7fd1344ba req-52a4d55f-100b-46ca-a17b-8b0ed5ab1672 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:38.345633 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c51bb62f-04df-45f5-803b-f1d7fd1344ba req-52a4d55f-100b-46ca-a17b-8b0ed5ab1672 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0845175 May 24 10:03:38.394738 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c51bb62f-04df-45f5-803b-f1d7fd1344ba req-827edac2-dd84-468d-960b-2c60a4d8f815 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420060 May 24 10:03:38.763152 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:03:39.012449 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79508e5d-9eeb-4c0f-aa4a-e3f62b9027ce req-b2bec400-eae6-4861-9330-5aaf63ba2294 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:39.013820 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79508e5d-9eeb-4c0f-aa4a-e3f62b9027ce req-b2bec400-eae6-4861-9330-5aaf63ba2294 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0867682 May 24 10:03:39.062439 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79508e5d-9eeb-4c0f-aa4a-e3f62b9027ce req-f6e1abb8-90ed-48cb-9520-712bf1e27b6d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429900 May 24 10:03:39.070440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fee4984-6459-416e-bea6-51243eeb0289 req-15e1f729-1864-4a8c-b6e7-d7265f45ff4d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:39.072062 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-15e1f729-1864-4a8c-b6e7-d7265f45ff4d service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.0872169 May 24 10:03:39.286198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fee4984-6459-416e-bea6-51243eeb0289 req-ec3acbd3-3593-4a13-b7b1-5633905cdcdf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:39.287539 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-ec3acbd3-3593-4a13-b7b1-5633905cdcdf service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.2055953 May 24 10:03:39.361825 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-a1b5b0c7-61fb-4791-891b-6e5131aa6803 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 217 time: 0.0663912 May 24 10:03:39.427296 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fee4984-6459-416e-bea6-51243eeb0289 req-c65b7253-73d7-42b3-b079-ca0d7e61ae92 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:39.428718 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-c65b7253-73d7-42b3-b079-ca0d7e61ae92 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.0593815 May 24 10:03:39.480098 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-7e0c5e55-9405-4f1e-a65d-93ca9e6d6534 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0429671 May 24 10:03:39.622952 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-44d1945f-672c-417f-aa9a-7fdf2384f230 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1348307 May 24 10:03:39.761394 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6fee4984-6459-416e-bea6-51243eeb0289 req-6c0771c3-6a02-4c59-b0b1-38b6c03e0d89 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1294305 May 24 10:03:40.255369 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc4390d4-831e-4f24-a3be-769a8d8e7dfd req-12148115-e7ff-4a3f-88c6-f2c64fbe743f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:40.256824 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fc4390d4-831e-4f24-a3be-769a8d8e7dfd req-12148115-e7ff-4a3f-88c6-f2c64fbe743f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0858958 May 24 10:03:40.303229 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fc4390d4-831e-4f24-a3be-769a8d8e7dfd req-a9857b99-f3dc-4a49-a779-ce2f8e7fed39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398536 May 24 10:03:41.497887 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1649b623-affa-43d2-a873-b66d0c999e03 req-0340f3e7-1ccf-4c47-9932-781c732baea1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:41.499351 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1649b623-affa-43d2-a873-b66d0c999e03 req-0340f3e7-1ccf-4c47-9932-781c732baea1 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0820563 May 24 10:03:41.545416 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1649b623-affa-43d2-a873-b66d0c999e03 req-1c494e25-f182-4e81-82ad-d617c23e5248 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402458 May 24 10:03:42.736107 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4cd502de-47fd-4e78-b624-86af9d20c0ec req-6d69c135-24fd-4e40-b351-9e095e7551ef tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:42.737571 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4cd502de-47fd-4e78-b624-86af9d20c0ec req-6d69c135-24fd-4e40-b351-9e095e7551ef tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0822332 May 24 10:03:42.783069 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4cd502de-47fd-4e78-b624-86af9d20c0ec req-5b5cff81-1a09-45fd-a6ca-4c43e34362a0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0395894 May 24 10:03:44.818651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86a46934-cb8e-4238-8ca8-27a29c40a266 req-6925d780-d60e-4196-8d2a-b4f6dc1405f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:44.820006 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-86a46934-cb8e-4238-8ca8-27a29c40a266 req-6925d780-d60e-4196-8d2a-b4f6dc1405f8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0835650 May 24 10:03:44.866103 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-86a46934-cb8e-4238-8ca8-27a29c40a266 req-80788205-d865-4822-b8a9-cf3035b11293 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399392 May 24 10:03:48.791030 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-602466e2-0a3c-4e0b-874d-ad073dc9b2ec service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0731339 May 24 10:03:48.940661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-6d330a53-6c05-43ad-8910-ae23e5d8753d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:48.942230 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-6d330a53-6c05-43ad-8910-ae23e5d8753d service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 1100 time: 0.0706079 May 24 10:03:49.091482 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-2e91bcfa-4d84-43f7-bab2-d198a95bcf44 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:49.092933 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-2e91bcfa-4d84-43f7-bab2-d198a95bcf44 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.1308634 May 24 10:03:49.200520 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-642d46b5-4ad2-4c3b-9039-997cb5e8249c service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 200 len: 1001 time: 0.0980337 May 24 10:03:49.266829 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-03e181c1-d59b-45d2-8af4-3e52692ed427 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:49.268297 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-03e181c1-d59b-45d2-8af4-3e52692ed427 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0611019 May 24 10:03:49.356589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-362407e7-4228-44e1-b772-8c1b2eaa3e36 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:03:49.358129 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-362407e7-4228-44e1-b772-8c1b2eaa3e36 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0743501 May 24 10:03:49.448156 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:03:49.502100 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-948477f5-3588-48d3-80d4-6d2f4b11ab3b service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1305618 May 24 10:03:49.634846 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf9e041e-7266-45e7-90af-099fdf52c6a9 req-0ce689d9-9469-42c6-899d-7df68da95edd service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1258192 May 24 10:03:50.162707 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:01.984966 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6651884-4da3-4a22-845e-ffe0e2e4b44d req-3d947f20-ef3a-49a4-ba12-d7d77ef64942 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:01.986463 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6651884-4da3-4a22-845e-ffe0e2e4b44d req-3d947f20-ef3a-49a4-ba12-d7d77ef64942 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.2380466 May 24 10:04:01.989122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0cd2b378-2dc5-4157-865c-2687365f835f req-4b0e26cf-dff2-4e05-b791-ca90f9d1a032 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:01.990477 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-4b0e26cf-dff2-4e05-b791-ca90f9d1a032 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.2024243 May 24 10:04:02.111239 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6651884-4da3-4a22-845e-ffe0e2e4b44d req-5a89eaaf-da15-4518-a4a2-8a6f16ac6156 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1187491 May 24 10:04:02.148729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0cd2b378-2dc5-4157-865c-2687365f835f req-625421d8-10d4-43d8-b452-2ab06f132a51 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:02.150169 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-625421d8-10d4-43d8-b452-2ab06f132a51 service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.1445875 May 24 10:04:02.222360 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-1b8426cd-cbb5-46d7-b68d-23e121e51c37 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 217 time: 0.0646620 May 24 10:04:02.285388 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0cd2b378-2dc5-4157-865c-2687365f835f req-4a92b415-0835-420a-bca6-a58dc1e7c9bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:02.286805 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-4a92b415-0835-420a-bca6-a58dc1e7c9bf service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.0570228 May 24 10:04:02.339067 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-4f9fbfd6-85ba-48cc-b4c9-3938aa0e0899 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0441623 May 24 10:04:02.473504 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-2b3a26c7-4bf4-4dbe-a4d4-817c8f521064 service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1261294 May 24 10:04:02.606575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0cd2b378-2dc5-4157-865c-2687365f835f req-ee58f4ef-6bdb-4e8f-90f2-0ad83b79c2ec service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1262054 May 24 10:04:03.265714 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-843cea41-68c1-4c74-8959-b92ca6a3aa29 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:03.296219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fa09a82-5d32-4ef2-9982-8a53b5458cfa req-cbf2bb99-b8a1-443c-be7a-23f9615238f9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:03.297709 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2fa09a82-5d32-4ef2-9982-8a53b5458cfa req-cbf2bb99-b8a1-443c-be7a-23f9615238f9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0792048 May 24 10:04:03.346625 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2fa09a82-5d32-4ef2-9982-8a53b5458cfa req-8130d1d9-1939-4f71-98dd-f51932fa154a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0431521 May 24 10:04:04.533137 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f343eab1-3220-4d89-bbd1-4f28a4259ba3 req-4e502796-03eb-4f47-b29a-dded02074d3b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:04.534615 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f343eab1-3220-4d89-bbd1-4f28a4259ba3 req-4e502796-03eb-4f47-b29a-dded02074d3b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0795970 May 24 10:04:04.579675 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f343eab1-3220-4d89-bbd1-4f28a4259ba3 req-8acb10c2-4dc4-4248-bb44-c9c722b13393 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392563 May 24 10:04:05.779618 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b64aaaad-e8e4-40cd-ad80-2a6aa33d5fc3 req-c9223e31-f646-4a7c-a149-dd700a2ed880 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:05.781189 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b64aaaad-e8e4-40cd-ad80-2a6aa33d5fc3 req-c9223e31-f646-4a7c-a149-dd700a2ed880 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0909817 May 24 10:04:05.827434 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b64aaaad-e8e4-40cd-ad80-2a6aa33d5fc3 req-d2ebee3b-952b-4789-b4e3-03e5cd534273 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403743 May 24 10:04:05.831455 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-03b3f62d-3cb3-44f3-bbcc-e1c1a328add9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:05.859934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:07.031606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b4392fc-192d-4a1b-9e19-46ed47b7d4b7 req-06b5faa5-a818-45f0-9bc3-4e63aac6e685 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:07.033225 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5b4392fc-192d-4a1b-9e19-46ed47b7d4b7 req-06b5faa5-a818-45f0-9bc3-4e63aac6e685 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0870204 May 24 10:04:07.142903 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5b4392fc-192d-4a1b-9e19-46ed47b7d4b7 req-46fabd73-027c-4b66-83f4-b29cc648f7af tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1036165 May 24 10:04:07.191197 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:08.339732 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db122c95-ecfe-409f-b8ee-1519b249d6ef req-cb9fe0a5-e485-4967-bfef-a4246eaeef29 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:08.341245 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-db122c95-ecfe-409f-b8ee-1519b249d6ef req-cb9fe0a5-e485-4967-bfef-a4246eaeef29 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0871730 May 24 10:04:08.385158 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-db122c95-ecfe-409f-b8ee-1519b249d6ef req-c2e7fe7f-e37e-4141-9f4e-01185005d724 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381706 May 24 10:04:08.784919 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:09.577374 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95bfdcd1-2df3-4ee6-9614-c2fc4a6e8188 req-f9d9e552-da96-43d2-be52-5bc5a247ee6d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:09.578851 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95bfdcd1-2df3-4ee6-9614-c2fc4a6e8188 req-f9d9e552-da96-43d2-be52-5bc5a247ee6d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0823503 May 24 10:04:09.624497 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-95bfdcd1-2df3-4ee6-9614-c2fc4a6e8188 req-a8027270-e140-487f-be08-aa07293530df tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399125 May 24 10:04:10.813504 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0ed81c8-2b8f-43cd-b715-b39e1d9a8c32 req-18c0f47b-86d8-4854-a781-c6e479a93394 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:10.815070 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0ed81c8-2b8f-43cd-b715-b39e1d9a8c32 req-18c0f47b-86d8-4854-a781-c6e479a93394 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0857339 May 24 10:04:10.859620 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0ed81c8-2b8f-43cd-b715-b39e1d9a8c32 req-be83ae66-3eb8-4fd3-ab3e-c942a7cdb7f9 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389705 May 24 10:04:12.046494 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2862e21c-70ad-4f1e-876d-2b9050ebcb04 req-97de3fa7-0b45-48c0-8580-5a8c44875d28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:12.049272 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2862e21c-70ad-4f1e-876d-2b9050ebcb04 req-97de3fa7-0b45-48c0-8580-5a8c44875d28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0825157 May 24 10:04:12.091563 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2862e21c-70ad-4f1e-876d-2b9050ebcb04 req-559a7dcd-96a8-4a1b-a8a7-f46aadb25059 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0376410 May 24 10:04:13.142396 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d9141f-b65b-4bf0-bb7f-f73325b9ea8d req-a59ea07b-be21-4636-8156-58390460d314 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:13.143969 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e0d9141f-b65b-4bf0-bb7f-f73325b9ea8d req-a59ea07b-be21-4636-8156-58390460d314 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.0928686 May 24 10:04:13.190258 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e0d9141f-b65b-4bf0-bb7f-f73325b9ea8d req-1433ed4a-5cd3-43a3-85eb-3568c4968aa1 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397599 May 24 10:04:13.370838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-339d9e61-e6a9-4116-8a7c-3d74a0164828 req-e9744cea-9108-4009-b36b-d25027e4fab2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:13.372322 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-339d9e61-e6a9-4116-8a7c-3d74a0164828 req-e9744cea-9108-4009-b36b-d25027e4fab2 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0864699 May 24 10:04:13.419601 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-339d9e61-e6a9-4116-8a7c-3d74a0164828 req-51a17665-efec-420a-aa2a-463e3b16c6a4 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404887 May 24 10:04:14.381604 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c48b56fd-d1ac-4557-a529-87622223b1bd req-fe16f062-03dc-454a-baf9-f061c5aa1aa2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:14.383107 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c48b56fd-d1ac-4557-a529-87622223b1bd req-fe16f062-03dc-454a-baf9-f061c5aa1aa2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.0837111 May 24 10:04:14.430789 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c48b56fd-d1ac-4557-a529-87622223b1bd req-35c9d8bc-1055-4ffe-bb30-987a06d971c4 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418963 May 24 10:04:14.616264 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e150ba7-aaa9-48b5-9f2a-200395459e34 req-2948c06e-5a8a-4b48-8f03-9ffdab4bb19b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:14.617763 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e150ba7-aaa9-48b5-9f2a-200395459e34 req-2948c06e-5a8a-4b48-8f03-9ffdab4bb19b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0879259 May 24 10:04:14.664039 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e150ba7-aaa9-48b5-9f2a-200395459e34 req-8ed3f969-8f5f-4752-ae1f-b7c779d16339 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0406415 May 24 10:04:15.284372 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-90fc9dc5-f533-4762-9bdf-68f99f24ee1b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:15.285807 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-90fc9dc5-f533-4762-9bdf-68f99f24ee1b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.0803177 May 24 10:04:15.432133 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Deleting port 855e6647-516e-462d-8d07-0f5cf828da14 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:04:15.433042 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:15.577551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] For port 855e6647-516e-462d-8d07-0f5cf828da14, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=855e6647-516e-462d-8d07-0f5cf828da14,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:15.577551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:15.629564 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32d2a38a-2bd0-4f36-b6ed-ec5e34ae1214 req-21968998-9a51-4714-befb-fb04970e585f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:15.631163 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-32d2a38a-2bd0-4f36-b6ed-ec5e34ae1214 req-21968998-9a51-4714-befb-fb04970e585f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 946 time: 0.0941474 May 24 10:04:15.682020 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-32d2a38a-2bd0-4f36-b6ed-ec5e34ae1214 req-d8519975-4948-43e3-bf5c-cc5af2d47aaf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0440233 May 24 10:04:15.800387 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Calling delete_port for 855e6647-516e-462d-8d07-0f5cf828da14 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:04:15.873683 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bbb34a5a-60fb-4b6d-90c2-ba8cac130459 req-6c91f9fb-d3e8-43d1-991f-0ee3b7879798 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:15.875239 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bbb34a5a-60fb-4b6d-90c2-ba8cac130459 req-6c91f9fb-d3e8-43d1-991f-0ee3b7879798 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0868571 May 24 10:04:15.921028 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bbb34a5a-60fb-4b6d-90c2-ba8cac130459 req-fbaa7f5a-024e-47e8-87e3-3b1e2dc7f688 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399351 May 24 10:04:15.985816 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:15.993293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:16.001364 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['855e6647-516e-462d-8d07-0f5cf828da14'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:04:16.048892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Pushing event deleted for resources: {'Port': ['ID=855e6647-516e-462d-8d07-0f5cf828da14,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:16.129301 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.129674 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.130030 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.130382 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.130757 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.131106 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.131448 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.131900 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.132132 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.132470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.132810 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.133188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:16.144039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:16.184495 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] No DVR routers for this DVR port 855e6647-516e-462d-8d07-0f5cf828da14 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:04:16.189128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:16.192524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'name': 'network-vif-deleted', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:04:16.224124 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51ce65f9-0248-4e87-9a52-531e5e421b68 req-4d5509d0-a293-4bd5-b163-dc542472657a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:16.225707 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-51ce65f9-0248-4e87-9a52-531e5e421b68 req-4d5509d0-a293-4bd5-b163-dc542472657a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1908255 May 24 10:04:16.261771 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '61900f34-f721-4a3d-ad8c-ff38d86984fc', 'name': 'network-vif-deleted', 'tag': '855e6647-516e-462d-8d07-0f5cf828da14', 'status': 'completed', 'code': 200} May 24 10:04:16.274021 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-27a99b13-8b64-4f9b-96e1-6a4d81d402e7 req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "DELETE /v2.0/ports/855e6647-516e-462d-8d07-0f5cf828da14 HTTP/1.1" status: 204 len: 173 time: 0.9791811 May 24 10:04:16.289630 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-51ce65f9-0248-4e87-9a52-531e5e421b68 req-38e1f2f6-29df-4cc0-a9f5-147238877667 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0570235 May 24 10:04:16.844954 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-59181f4e-bd3f-47ec-a4ec-70b351bb9111 req-6dbc967a-e5db-4c3f-9da4-882b8b4677b1 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=61900f34-f721-4a3d-ad8c-ff38d86984fc HTTP/1.1" status: 200 len: 210 time: 0.0440154 May 24 10:04:17.115940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6e4bc6b-c8fd-4701-924a-84b901d2ec00 req-641e9bf4-9fe3-4fbe-99c3-1ac54fb82f82 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:17.117405 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f6e4bc6b-c8fd-4701-924a-84b901d2ec00 req-641e9bf4-9fe3-4fbe-99c3-1ac54fb82f82 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0804167 May 24 10:04:17.163575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f6e4bc6b-c8fd-4701-924a-84b901d2ec00 req-1aded23a-598d-46c9-93b9-77c73eddabf0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400345 May 24 10:04:17.484197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f318212e-9c43-41de-a0a9-5e4a0ecc02eb req-4ae868e5-ccea-4c3d-b7cb-0f13c531e76f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:17.485698 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f318212e-9c43-41de-a0a9-5e4a0ecc02eb req-4ae868e5-ccea-4c3d-b7cb-0f13c531e76f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0798008 May 24 10:04:17.534103 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f318212e-9c43-41de-a0a9-5e4a0ecc02eb req-26e27f6a-c8de-4ecc-9b30-cee72e3fc827 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0426617 May 24 10:04:18.500197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4196ace8-dd61-44da-b929-ce1393f1415f req-a808f636-a5b5-4bb7-916e-1f3289580064 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:18.501790 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4196ace8-dd61-44da-b929-ce1393f1415f req-a808f636-a5b5-4bb7-916e-1f3289580064 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0832946 May 24 10:04:18.573565 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4196ace8-dd61-44da-b929-ce1393f1415f req-5445d366-dcd6-4228-b286-1995c7179e30 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398481 May 24 10:04:18.728544 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-266749e1-7be8-418d-99c6-d6048f76d269 req-4689acf3-939f-4b22-b50d-a1e4f8d9c9d8 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:18.730808 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-266749e1-7be8-418d-99c6-d6048f76d269 req-4689acf3-939f-4b22-b50d-a1e4f8d9c9d8 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0876176 May 24 10:04:18.777974 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-266749e1-7be8-418d-99c6-d6048f76d269 req-c90b57c4-fd50-46c6-8244-95b244937789 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407183 May 24 10:04:19.238984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 56914) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:19.373213 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.388372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:19.392134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Usage tracker for resource:security_group_rule and tenant:f6bf422f5e4e4139b7679d291ead94a8 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:19.394915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:19.400473 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:19.483085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.562476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.569976 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-458bbd8f-2d10-48ec-b287-680382b8b2d3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:19.613593 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=f6bf422f5e4e4139b7679d291ead94a8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3734939 May 24 10:04:19.614572 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9cf4ff0f-57f8-49e1-b9a2-61e6d54e7fe8,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:19.615543 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 56924) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:19.622864 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.623219 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.623635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.623981 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.624316 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.624652 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.624988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.625347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.625721 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.626009 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.626338 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.626805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:19.631456 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:19.689265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.716038 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.740046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64084f68-df6a-4659-8b5a-a2587f193c42 req-f868da83-d80a-411f-af18-4d2e1c1c5ca0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:19.741557 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-64084f68-df6a-4659-8b5a-a2587f193c42 req-f868da83-d80a-411f-af18-4d2e1c1c5ca0 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0843444 May 24 10:04:19.786975 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-64084f68-df6a-4659-8b5a-a2587f193c42 req-6be73d1c-21c8-49f5-bb47-48894efba795 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396712 May 24 10:04:19.825313 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.829012 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bdc07bbd-6441-4d85-9a1b-08515f2f792c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:19.842708 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9cf4ff0f-57f8-49e1-b9a2-61e6d54e7fe8,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:19.845678 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:19.849123 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:19.851306 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/9cf4ff0f-57f8-49e1-b9a2-61e6d54e7fe8 HTTP/1.1" status: 204 len: 168 time: 0.2346520 May 24 10:04:19.967524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6296ee6f-0e44-4a6b-9002-1ea05b29073d req-0001241b-3825-4372-a46c-271c599fce2c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:19.968998 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6296ee6f-0e44-4a6b-9002-1ea05b29073d req-0001241b-3825-4372-a46c-271c599fce2c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0815790 May 24 10:04:20.013982 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6296ee6f-0e44-4a6b-9002-1ea05b29073d req-24b4e909-bdfa-4314-85a8-893ddf78b8bb tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0393186 May 24 10:04:20.088021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:20.703818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 855e6647-516e-462d-8d07-0f5cf828da14 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:04:20.737316 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 855e6647-516e-462d-8d07-0f5cf828da14 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:04:20.737683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 855e6647-516e-462d-8d07-0f5cf828da14 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:04:20.978884 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70dcce10-7a6d-41ec-b2bd-2e26663ec868 req-e089076a-ab87-41f9-860b-a2be55c9b4e5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:20.980463 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-70dcce10-7a6d-41ec-b2bd-2e26663ec868 req-e089076a-ab87-41f9-860b-a2be55c9b4e5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0850463 May 24 10:04:21.026507 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-70dcce10-7a6d-41ec-b2bd-2e26663ec868 req-bae8f0c6-3974-40cf-923b-e095b2773db8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402377 May 24 10:04:21.207428 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83294e69-9134-4ad2-8145-a769e4c813bd req-0b611006-fc8e-4933-bc86-c0dd431166a3 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:21.209010 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-83294e69-9134-4ad2-8145-a769e4c813bd req-0b611006-fc8e-4933-bc86-c0dd431166a3 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0873923 May 24 10:04:21.254854 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-83294e69-9134-4ad2-8145-a769e4c813bd req-4464cfc3-b84d-4176-95a9-dc201579d8e5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0402143 May 24 10:04:22.215871 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d9d2f28-d58c-45ef-b191-07d988438dc4 req-c55765ee-d6d8-4481-b908-2fc48c1d54bb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:22.217397 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5d9d2f28-d58c-45ef-b191-07d988438dc4 req-c55765ee-d6d8-4481-b908-2fc48c1d54bb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0823531 May 24 10:04:22.266345 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5d9d2f28-d58c-45ef-b191-07d988438dc4 req-d25cbde4-77bf-4eae-aad0-918656c635dc tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425365 May 24 10:04:22.449480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6988873-66ee-4d35-b181-06c5afe03a0e req-fabb9932-5f99-418f-8341-4247c638b275 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:22.450947 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b6988873-66ee-4d35-b181-06c5afe03a0e req-fabb9932-5f99-418f-8341-4247c638b275 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0857871 May 24 10:04:22.494130 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b6988873-66ee-4d35-b181-06c5afe03a0e req-0877a657-d3c1-4bca-b651-4a8a73dc232d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0373046 May 24 10:04:23.454061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fc516a0-e2ea-4010-85f8-93156dbbf82f req-a75b3f7a-c6c3-40f6-b1a8-50cff6008cdc tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:23.455530 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6fc516a0-e2ea-4010-85f8-93156dbbf82f req-a75b3f7a-c6c3-40f6-b1a8-50cff6008cdc tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0819016 May 24 10:04:23.500787 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6fc516a0-e2ea-4010-85f8-93156dbbf82f req-6c4f0ed3-75ee-46b6-8265-a77082bcc896 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394185 May 24 10:04:23.690485 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a462860-2b6c-4207-a864-27d0a2be3e61 req-5b87fb0f-f3a6-4a60-861b-13724706ac13 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:23.692116 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5a462860-2b6c-4207-a864-27d0a2be3e61 req-5b87fb0f-f3a6-4a60-861b-13724706ac13 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0888538 May 24 10:04:23.740842 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5a462860-2b6c-4207-a864-27d0a2be3e61 req-f5572753-4456-43af-b1bf-006fcc3af465 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0424314 May 24 10:04:24.024105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb3c8edb-b14a-4236-a393-8fb11381e474 req-9c35dd50-901c-4ea6-80d7-d7ada62b6b6f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:24.025579 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bb3c8edb-b14a-4236-a393-8fb11381e474 req-9c35dd50-901c-4ea6-80d7-d7ada62b6b6f service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.0679793 May 24 10:04:24.076010 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bb3c8edb-b14a-4236-a393-8fb11381e474 req-83786264-d18e-4274-a70c-c4a6c9ebe481 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0380781 May 24 10:04:24.693779 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0b20e2b-0eb9-409b-9573-54d9c0d63c86 req-e3fb6400-2726-449f-8425-7b47d334413c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:24.695328 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d0b20e2b-0eb9-409b-9573-54d9c0d63c86 req-e3fb6400-2726-449f-8425-7b47d334413c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0837851 May 24 10:04:24.741066 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d0b20e2b-0eb9-409b-9573-54d9c0d63c86 req-d6978a2b-a13a-43e0-bc5f-097d704574ff tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394444 May 24 10:04:24.924639 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f2b4f0-9a4d-4fe9-b70f-fc5c5d7ace5f req-f2f0bf06-e9ef-497a-9b9d-895b69dbef6b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:24.926077 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-92f2b4f0-9a4d-4fe9-b70f-fc5c5d7ace5f req-f2f0bf06-e9ef-497a-9b9d-895b69dbef6b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0805764 May 24 10:04:24.971099 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-92f2b4f0-9a4d-4fe9-b70f-fc5c5d7ace5f req-9765541e-fc6c-4865-8bc3-4b73e39eee9c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394337 May 24 10:04:25.932457 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d8b2d7c8-c42a-45db-b91f-97082a75ee9d req-6d7d97f8-35a1-4b16-b0e5-d662d4797afa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:25.934518 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d8b2d7c8-c42a-45db-b91f-97082a75ee9d req-6d7d97f8-35a1-4b16-b0e5-d662d4797afa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0777204 May 24 10:04:25.977930 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d8b2d7c8-c42a-45db-b91f-97082a75ee9d req-be97d039-6ed4-4b5f-98ff-b993393d6a45 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379238 May 24 10:04:26.156674 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-701b2bee-85fc-4913-ab1c-bca2a79b59e8 req-096c57da-7d64-4362-918d-ca17a53f772c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:26.158068 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-701b2bee-85fc-4913-ab1c-bca2a79b59e8 req-096c57da-7d64-4362-918d-ca17a53f772c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0825794 May 24 10:04:26.202220 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-701b2bee-85fc-4913-ab1c-bca2a79b59e8 req-3b681bed-a2d0-454b-b160-939977317ce4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386479 May 24 10:04:26.449036 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:04:27.165279 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3923d0e0-5cf9-4b98-9933-e39ab995ecdf req-833d55c9-f6ed-49c2-b579-a7ef454cdf3b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:27.166994 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3923d0e0-5cf9-4b98-9933-e39ab995ecdf req-833d55c9-f6ed-49c2-b579-a7ef454cdf3b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0818741 May 24 10:04:27.210524 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3923d0e0-5cf9-4b98-9933-e39ab995ecdf req-5a7f5788-1030-4fa7-bf5d-6bd1f9a90a5f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0373261 May 24 10:04:27.385469 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01de659a-43ac-44f0-9234-93935d6c842c req-ee22070b-3d5c-453f-830a-e5858f49a8a5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:27.387427 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01de659a-43ac-44f0-9234-93935d6c842c req-ee22070b-3d5c-453f-830a-e5858f49a8a5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0792854 May 24 10:04:27.431557 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01de659a-43ac-44f0-9234-93935d6c842c req-30bf7d3b-7d5a-4790-8bc3-17bd976266b9 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384262 May 24 10:04:28.398291 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a95400ad-e2a4-4e0a-aee4-1ede6dd556c6 req-676aa834-fdab-4a6e-9021-68bbbc048d13 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:28.399760 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a95400ad-e2a4-4e0a-aee4-1ede6dd556c6 req-676aa834-fdab-4a6e-9021-68bbbc048d13 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0806053 May 24 10:04:28.444267 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a95400ad-e2a4-4e0a-aee4-1ede6dd556c6 req-37106e62-a202-4089-8cd3-4722a4001d0e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0383031 May 24 10:04:28.620828 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-347d1f41-463f-4e83-a3f9-0b9869faa2e9 req-d1c3d176-2722-4ea7-ba74-628bbc72c611 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:28.622181 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-347d1f41-463f-4e83-a3f9-0b9869faa2e9 req-d1c3d176-2722-4ea7-ba74-628bbc72c611 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0848048 May 24 10:04:28.666913 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-347d1f41-463f-4e83-a3f9-0b9869faa2e9 req-29a23faf-b25d-4583-9963-629e5e4eca83 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0389991 May 24 10:04:29.639130 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60be791-e099-4630-a901-7b8a615e674d req-e7bade07-5321-4b7f-b340-62706ffbfdc8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:29.640582 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60be791-e099-4630-a901-7b8a615e674d req-e7bade07-5321-4b7f-b340-62706ffbfdc8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0866420 May 24 10:04:29.685388 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60be791-e099-4630-a901-7b8a615e674d req-e9cfa4d0-3983-413a-acb9-63b5a05419b7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390327 May 24 10:04:29.854736 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00a965cb-ccb0-49a3-a100-3c4c91de8352 req-c5eaae16-f6cf-45c8-9fd2-7e4eada67d3f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:29.856759 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-00a965cb-ccb0-49a3-a100-3c4c91de8352 req-c5eaae16-f6cf-45c8-9fd2-7e4eada67d3f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0800490 May 24 10:04:29.899867 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-00a965cb-ccb0-49a3-a100-3c4c91de8352 req-afa74bcd-dae6-4064-99dd-7a5c26ce18b4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0374146 May 24 10:04:30.869445 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46b1d06e-f8b7-4f3a-8b31-831de44241d3 req-652e31b2-94ff-4f23-9e7a-22388bb58084 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:30.870902 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46b1d06e-f8b7-4f3a-8b31-831de44241d3 req-652e31b2-94ff-4f23-9e7a-22388bb58084 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0805860 May 24 10:04:30.972217 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-46b1d06e-f8b7-4f3a-8b31-831de44241d3 req-31d094a3-c012-4f50-bfb4-1034fffa5c78 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0952058 May 24 10:04:31.151205 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5fd9db63-ecda-4417-bb27-4ccdd25a3ca9 req-6ccdd963-93ef-4c37-9620-560fab6debb4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:31.152676 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5fd9db63-ecda-4417-bb27-4ccdd25a3ca9 req-6ccdd963-93ef-4c37-9620-560fab6debb4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0803025 May 24 10:04:31.199247 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5fd9db63-ecda-4417-bb27-4ccdd25a3ca9 req-42e81def-a0ba-4f1a-96ca-3981885fedb0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0404859 May 24 10:04:31.231987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-43ae504a-1401-4702-973c-bbe712fa7b63 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:31.233460 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-43ae504a-1401-4702-973c-bbe712fa7b63 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/networks?id=2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 200 len: 813 time: 0.2436547 May 24 10:04:31.248573 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-151197cf-bdec-43d6-b266-5fa7266a307e tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/quotas/0fa87500244642ee805dba47191bc7e6 HTTP/1.1" status: 200 len: 375 time: 0.0095675 May 24 10:04:31.293667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-75efc1be-5c34-4699-a33e-a79d543f2bff tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:31.373532 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-75efc1be-5c34-4699-a33e-a79d543f2bff tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:31.374802 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-75efc1be-5c34-4699-a33e-a79d543f2bff tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1213210 May 24 10:04:31.510021 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-ba43416f-0726-4e19-8b92-744f77013bdf service neutron] 10.222.0.67 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1282067 May 24 10:04:31.640058 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-5a1cf9a3-4e49-414f-b82a-9a888f2c0e25 service neutron] 10.222.0.67 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1243145 May 24 10:04:31.867348 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9c17a130-bb95-47b0-8f13-bfc04f480c1e req-b4a02660-90d1-470a-b979-d80ec0ef4d6f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0446224 May 24 10:04:32.160487 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf953093-e35d-49b7-b7f0-4abb35795d51 req-03f57da6-1c26-48c5-ba19-3934de753800 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:32.161967 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf953093-e35d-49b7-b7f0-4abb35795d51 req-03f57da6-1c26-48c5-ba19-3934de753800 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0792532 May 24 10:04:32.211342 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bf953093-e35d-49b7-b7f0-4abb35795d51 req-748e748c-2cc9-486a-99d3-b1877f8d6201 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0431728 May 24 10:04:32.399651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-115b6e7d-cdcc-4573-ac17-a8bc55db24c2 req-b58e6b5a-e524-429f-9fe1-d6224a1125ad tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:32.399651 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-115b6e7d-cdcc-4573-ac17-a8bc55db24c2 req-b58e6b5a-e524-429f-9fe1-d6224a1125ad tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0874898 May 24 10:04:32.442047 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-115b6e7d-cdcc-4573-ac17-a8bc55db24c2 req-19ffe093-8cf9-43e0-b1b4-dfd03028fbc0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0408001 May 24 10:04:33.012743 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e4750153-e26f-4d02-a85e-91a1f076b0d5 req-f7280c58-d192-4b25-b32a-2b93dcfaaf61 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0435460 May 24 10:04:33.276704 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d98c1a03-b8c0-4bbd-a60c-93e5dcf0564d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:33.412599 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-661e4202-2133-4e0f-ae9f-16d866cc366b req-1079d384-c716-4632-acf3-d565f0a51d0b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:33.414919 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-661e4202-2133-4e0f-ae9f-16d866cc366b req-1079d384-c716-4632-acf3-d565f0a51d0b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0957747 May 24 10:04:33.460547 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-661e4202-2133-4e0f-ae9f-16d866cc366b req-66051908-1664-4685-9f60-3aa887c5c53e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397456 May 24 10:04:33.636505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70f32201-6508-444c-aaac-f0a49043b762 req-d7d379c5-6c26-4dbf-b1b1-c2ba2a06a66a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:33.638196 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-70f32201-6508-444c-aaac-f0a49043b762 req-d7d379c5-6c26-4dbf-b1b1-c2ba2a06a66a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0839660 May 24 10:04:33.687718 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-70f32201-6508-444c-aaac-f0a49043b762 req-066a482e-b65d-40b3-ad21-4897ce825130 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0422587 May 24 10:04:34.168211 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e686c010-4af3-41df-b870-c03d7ce245f0 req-ca4db0b0-e098-46dd-a608-955f11485a2b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0409684 May 24 10:04:34.655856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6be00f5a-4bbf-4545-bfa1-b6535ebda617 req-46f50f9e-b7c5-418c-8fe5-25150971376d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:34.657385 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6be00f5a-4bbf-4545-bfa1-b6535ebda617 req-46f50f9e-b7c5-418c-8fe5-25150971376d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0848737 May 24 10:04:34.703480 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6be00f5a-4bbf-4545-bfa1-b6535ebda617 req-4076288e-b49b-43a6-8c9e-5ea3eb185048 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402422 May 24 10:04:34.879986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39541b47-ce33-457b-b705-88e3d9669a91 req-e6131282-32d9-4b4c-8cec-77ae708d7ac1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:34.881496 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-39541b47-ce33-457b-b705-88e3d9669a91 req-e6131282-32d9-4b4c-8cec-77ae708d7ac1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0860665 May 24 10:04:34.928457 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-39541b47-ce33-457b-b705-88e3d9669a91 req-a93f714d-a26a-46d2-9c90-5e363637dfeb tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0414567 May 24 10:04:35.340919 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5a6367be-39a0-4686-9d26-bc3c1e4389ad req-848a36c7-76d6-4c4e-9e33-ae02553219d1 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0435562 May 24 10:04:35.410201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ade9777b-5c54-4e66-82b8-a76606536129 req-fefce400-68f6-4970-86d5-f66e2e7000c8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:35.411677 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ade9777b-5c54-4e66-82b8-a76606536129 req-fefce400-68f6-4970-86d5-f66e2e7000c8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0812943 May 24 10:04:35.456980 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ade9777b-5c54-4e66-82b8-a76606536129 req-27647624-5ca6-4728-91a8-1cf3056997ac tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392947 May 24 10:04:35.518385 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c2f8264-f3dc-453a-848a-63e031256069 req-61429b43-2835-4d21-9d5b-ac830f138f79 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:35.519842 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1c2f8264-f3dc-453a-848a-63e031256069 req-61429b43-2835-4d21-9d5b-ac830f138f79 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0850916 May 24 10:04:35.565573 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1c2f8264-f3dc-453a-848a-63e031256069 req-823f9f8a-d005-46bd-94cc-8271aa254a23 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0401554 May 24 10:04:35.813324 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5b207f21-be4f-4c05-8f1d-56d96d0cbde1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:35.844207 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:36.493099 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c9ddf618-b56d-4b66-bc5a-f71be3aee8f0 req-284fe2e8-9ef2-4bfc-ac4f-26852cd8f685 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0456922 May 24 10:04:36.647262 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a26dedd-6245-4f7b-af9e-3d8fa0d454c7 req-47095aa2-76e1-42a5-ae43-bbb14502951e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:36.647262 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0a26dedd-6245-4f7b-af9e-3d8fa0d454c7 req-47095aa2-76e1-42a5-ae43-bbb14502951e tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0795395 May 24 10:04:36.694456 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0a26dedd-6245-4f7b-af9e-3d8fa0d454c7 req-e98e47f5-c40a-4cd0-a4ff-5915d7646fba tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427554 May 24 10:04:36.751381 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d3e8aec-65b6-42c3-aacb-095f4daa6d29 req-329279f0-815e-4b94-a7ca-2e756ff605a4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:36.752875 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1d3e8aec-65b6-42c3-aacb-095f4daa6d29 req-329279f0-815e-4b94-a7ca-2e756ff605a4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0817292 May 24 10:04:36.798090 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1d3e8aec-65b6-42c3-aacb-095f4daa6d29 req-122bc6b8-edf5-4614-acaa-090dc83ab0ae tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0389235 May 24 10:04:37.184933 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:37.444034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14109958-e702-4024-8765-a1b141c1dbd2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:37.445354 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14109958-e702-4024-8765-a1b141c1dbd2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "GET /v2.0/networks?id=2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 200 len: 813 time: 0.1564379 May 24 10:04:37.449113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-c08f4857-6157-4d97-9cbf-40ac84d3ef4b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:37.450554 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-c08f4857-6157-4d97-9cbf-40ac84d3ef4b service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=b08409afad374227ace1cdf31e4e9412&device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 1098 time: 0.0772972 May 24 10:04:37.469341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Request body: {'port': {'device_id': '8315e706-66b1-49e6-a581-061abaf6c6db', 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18', 'admin_state_up': True, 'tenant_id': '0fa87500244642ee805dba47191bc7e6'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:37.617702 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:37.663216 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-88940b1d-2541-4591-8a73-b76500434442 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:37.665003 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:37.665493 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-88940b1d-2541-4591-8a73-b76500434442 service neutron] 10.222.0.95 "GET /v2.0/networks?id=b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 200 len: 918 time: 0.2009320 May 24 10:04:37.675015 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e9836783-6871-4fc5-97c1-bae6202b5bf2 req-545e5e7a-9315-4ed1-80cd-8f3f75badeec tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0529616 May 24 10:04:37.739789 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-1fb38b00-2723-41eb-8d09-05a00ac8a6fd service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 200 len: 217 time: 0.0672431 May 24 10:04:37.748696 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2008800-c33b-45af-8032-9adb8649fe40 req-f038205c-7745-4f15-a527-8162dc0013af service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:37.750123 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-f038205c-7745-4f15-a527-8162dc0013af service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.1650121 May 24 10:04:37.768293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Made reservation on behalf of 0fa87500244642ee805dba47191bc7e6 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:04:37.772356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:37.949299 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-ecafd09b-852d-4abf-b2d0-4c69ad762a01 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:37.950751 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-ecafd09b-852d-4abf-b2d0-4c69ad762a01 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 200 len: 858 time: 0.2042964 May 24 10:04:37.987575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Allocated IP 172.20.0.3 (2852c3dc-6508-45a7-9576-a36787475d18/9ee9ac65-ecfe-47d4-bbac-4561dabf43e6/086b9a1c-9348-4b08-b6f7-f934b5e8aaa4) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:04:38.010935 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-13e1a28c-4561-46e9-b576-b29d24fc4c72 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=b0fff367-5ea3-4833-b1bf-44e4dd284af1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0513370 May 24 10:04:38.077595 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2008800-c33b-45af-8032-9adb8649fe40 req-cc4b7aa9-61c3-48ed-b805-7ab1d0291937 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.079061 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-cc4b7aa9-61c3-48ed-b805-7ab1d0291937 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.2964005 May 24 10:04:38.092408 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8286d3f-4720-4e52-87da-78fb1632cde8 req-535b0c02-b0ea-48f9-9b6f-f0a75b65e71c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.094021 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a8286d3f-4720-4e52-87da-78fb1632cde8 req-535b0c02-b0ea-48f9-9b6f-f0a75b65e71c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.2848122 May 24 10:04:38.206537 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e5deca4-910c-4b0c-abb5-836281945ef0 req-9f57e872-1c1f-4b09-a908-90f5319ee170 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.209273 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e5deca4-910c-4b0c-abb5-836281945ef0 req-9f57e872-1c1f-4b09-a908-90f5319ee170 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2647655 May 24 10:04:38.342133 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e5deca4-910c-4b0c-abb5-836281945ef0 req-8c44ba42-1466-4034-98c2-1cbb99c6e70b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0824878 May 24 10:04:38.365447 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a8286d3f-4720-4e52-87da-78fb1632cde8 req-f8f43dea-e4c2-4a43-bd6d-611a191ce4ee tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2273090 May 24 10:04:38.366796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:38.399504 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-60345a06-d06f-43fa-8d79-0af3139f4106 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1000 time: 0.3044865 May 24 10:04:38.405714 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-2a77c1aa-2135-4ab8-9231-8ec77f8756da service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3862088 May 24 10:04:38.550234 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2008800-c33b-45af-8032-9adb8649fe40 req-8acb2151-7eda-49c9-a672-518dac2d38a8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.551688 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-8acb2151-7eda-49c9-a672-518dac2d38a8 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.1450958 May 24 10:04:38.574525 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:38.584595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:38.644144 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1dca1a2e-1a9e-4b79-8d11-ccf136c38425 req-94fb69a8-d762-4f8c-a020-0cd1cc88169a service neutron] 10.222.0.95 "GET /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2058423 May 24 10:04:38.654935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Network 2852c3dc-6508-45a7-9576-a36787475d18 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:38.658770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:38.658931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2008800-c33b-45af-8032-9adb8649fe40 req-98c39550-d1d2-42c3-942e-6e581fb81d4e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.661085 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-98c39550-d1d2-42c3-942e-6e581fb81d4e service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1012347 May 24 10:04:38.663243 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:04:38.751263 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:38.764188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.765826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:38.790067 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Pushing event updated for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:38.805585 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:38.822906 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-980ef009-dfa4-432b-aa39-28aae9d0fc95 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1551909 May 24 10:04:38.936089 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d70e721-70b8-4830-a0ac-338bd68812b7 req-eda1ff79-a3b4-45b9-a8c7-03fd2ab50857 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:38.937004 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6d70e721-70b8-4830-a0ac-338bd68812b7 req-eda1ff79-a3b4-45b9-a8c7-03fd2ab50857 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:38.952074 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4895768 May 24 10:04:38.964493 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Request body: {'port': {'device_id': '8315e706-66b1-49e6-a581-061abaf6c6db', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:38.989036 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a2008800-c33b-45af-8032-9adb8649fe40 req-75b8f21e-f639-4d98-b2dc-3d7a39d32e83 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1514180 May 24 10:04:39.049680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6d70e721-70b8-4830-a0ac-338bd68812b7 req-eda1ff79-a3b4-45b9-a8c7-03fd2ab50857 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:39.050919 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6d70e721-70b8-4830-a0ac-338bd68812b7 req-eda1ff79-a3b4-45b9-a8c7-03fd2ab50857 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 931 time: 0.2234807 May 24 10:04:39.148635 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6d70e721-70b8-4830-a0ac-338bd68812b7 req-7caae543-def7-4f2f-9877-965fcde06bc7 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0860951 May 24 10:04:39.157531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:39.568017 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ec0b72c-4944-44b9-8af1-af43aeba90c9 req-22313657-43dd-4a44-ad8a-f21c62eb19cf tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:39.569597 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8ec0b72c-4944-44b9-8af1-af43aeba90c9 req-22313657-43dd-4a44-ad8a-f21c62eb19cf tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0966885 May 24 10:04:39.573735 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:04:39.586455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:39.607856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Transition to ACTIVE for port object 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:39.619241 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8ec0b72c-4944-44b9-8af1-af43aeba90c9 req-b5b1f365-7656-4eff-8716-30278b3b3a81 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432911 May 24 10:04:39.620728 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d3341e0a-bdbb-4069-b9f2-f235a9db237a req-da4de92b-955d-4080-b4db-592f978891fa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:39.622263 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d3341e0a-bdbb-4069-b9f2-f235a9db237a req-da4de92b-955d-4080-b4db-592f978891fa tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1300046 May 24 10:04:39.670959 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d3341e0a-bdbb-4069-b9f2-f235a9db237a req-daf7c54c-44f6-4841-bb6b-fee2daecc4b3 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0421026 May 24 10:04:39.756600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:39.760806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:39.886872 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Pushing event updated for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:39.916821 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:39.992992 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Attempting to bind port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:04:39.993347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Attempting to bind port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:04:39.993669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Attempting to bind port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 on network 2852c3dc-6508-45a7-9576-a36787475d18 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:04:39.998863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 on network 2852c3dc-6508-45a7-9576-a36787475d18 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:04:39.999177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Attempting to bind port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 on network 2852c3dc-6508-45a7-9576-a36787475d18 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:04:40.004311 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:04:40.004642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Checking segment: {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:40.005023 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Bound using segment: {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:04:40.005477 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Bound port: 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '3ddab603-0e7a-4025-8390-7039b3ef3fd1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 63, 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:04:40.005822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:40.095564 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:04:40.098687 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:04:40.162657 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:40.167002 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:40.225415 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.226031 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:40.227395 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] 10.222.0.95 "PUT /v2.0/ports/086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 HTTP/1.1" status: 200 len: 1093 time: 1.2677763 May 24 10:04:40.227892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:04:40.250826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Pushing event updated for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:40.261953 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:40.283079 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4', 'status': 'completed', 'code': 200} May 24 10:04:40.356934 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4be1ae3b-8920-4790-9938-a0e0067778ef req-33a5dcd9-cc91-4396-b348-d82567baa179 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.359132 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4be1ae3b-8920-4790-9938-a0e0067778ef req-33a5dcd9-cc91-4396-b348-d82567baa179 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 943 time: 0.1012409 May 24 10:04:40.397023 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-481fb658-34c8-4414-a2c3-a25b851d7e05 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.398596 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-481fb658-34c8-4414-a2c3-a25b851d7e05 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 1096 time: 0.0790784 May 24 10:04:40.418667 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4be1ae3b-8920-4790-9938-a0e0067778ef req-e52fb5b1-4288-47f2-8bb1-745e212b3e2a tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0495753 May 24 10:04:40.578397 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-4a967eac-b216-4676-a1b2-3cb2d39ca566 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.579914 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-4a967eac-b216-4676-a1b2-3cb2d39ca566 service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=0fa87500244642ee805dba47191bc7e6&shared=False HTTP/1.1" status: 200 len: 913 time: 0.1725388 May 24 10:04:40.863627 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35120663-07d2-47b1-a6d7-44b6fe9f50d7 req-7edc8801-a872-4b16-b9d3-6d423842387c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.865137 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35120663-07d2-47b1-a6d7-44b6fe9f50d7 req-7edc8801-a872-4b16-b9d3-6d423842387c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0808029 May 24 10:04:40.866043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-c29794e6-72ea-43f6-8c59-8da7c45ce9c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.867476 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5447fd5d-f1d9-41c5-b237-1d4c7defa3e0 req-c29794e6-72ea-43f6-8c59-8da7c45ce9c6 service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.2774792 May 24 10:04:40.889032 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c35633-617a-4298-a4da-156127935e58 req-3dabf93d-8943-49d5-a1ea-13985bb06590 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:40.890546 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c7c35633-617a-4298-a4da-156127935e58 req-3dabf93d-8943-49d5-a1ea-13985bb06590 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.1229115 May 24 10:04:40.918754 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-35120663-07d2-47b1-a6d7-44b6fe9f50d7 req-017c205f-1177-4bd3-81df-2f82dc445513 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0474823 May 24 10:04:40.989462 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c7c35633-617a-4298-a4da-156127935e58 req-5485b0cb-60b2-4081-bbcd-3edcecbd8ee0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0916660 May 24 10:04:41.007292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-3f168e3b-9a62-4d0c-9091-ad8b867317cd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:41.008766 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-3f168e3b-9a62-4d0c-9091-ad8b867317cd service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=0fa87500244642ee805dba47191bc7e6&device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 1096 time: 0.0978301 May 24 10:04:41.081120 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-008b8b7c-e627-4db6-b869-6d8487293a9e service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 HTTP/1.1" status: 200 len: 217 time: 0.0635657 May 24 10:04:41.144419 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-4f702a9c-5a99-4415-a558-1e5444800025 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:41.149118 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-4f702a9c-5a99-4415-a558-1e5444800025 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=9ee9ac65-ecfe-47d4-bbac-4561dabf43e6 HTTP/1.1" status: 200 len: 853 time: 0.0585752 May 24 10:04:41.195458 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-91713df6-d8c1-4d60-bf25-c229d5374363 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=2852c3dc-6508-45a7-9576-a36787475d18&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0425406 May 24 10:04:41.329894 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-a5854d73-6c72-4def-afd0-1bbecd74a724 service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1279709 May 24 10:04:41.464618 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d70999f1-ded7-4e91-b157-8981ee5b592a req-e1a70e93-ba8c-47e4-96ff-99688cb64f36 service neutron] 10.222.0.95 "GET /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1277514 May 24 10:04:41.634536 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1f195ea-f58c-4ce0-a69d-f023c4af0aa3 req-6c95a91c-8caa-4194-b4fc-b6a51a64e618 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:41.640411 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e1f195ea-f58c-4ce0-a69d-f023c4af0aa3 req-6c95a91c-8caa-4194-b4fc-b6a51a64e618 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 943 time: 0.0970180 May 24 10:04:41.686107 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e1f195ea-f58c-4ce0-a69d-f023c4af0aa3 req-aeb259d1-0920-49e2-8fd6-06a941e9acfa tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418489 May 24 10:04:42.923339 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-585dbb9b-6232-42f2-af3e-1ea2ab392e25 req-68401faa-9b67-49e2-8e99-cd13daff2a76 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:42.925207 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-585dbb9b-6232-42f2-af3e-1ea2ab392e25 req-68401faa-9b67-49e2-8e99-cd13daff2a76 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 943 time: 0.1212869 May 24 10:04:42.973428 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-585dbb9b-6232-42f2-af3e-1ea2ab392e25 req-dfbc7255-1582-4e46-aee3-608803c6666c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416393 May 24 10:04:44.163283 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1299421f-c08b-437e-8fe4-d6bfc5572086 req-c64a151a-4bfd-4954-8ea6-5548cfa6aa0b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:44.164922 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1299421f-c08b-437e-8fe4-d6bfc5572086 req-c64a151a-4bfd-4954-8ea6-5548cfa6aa0b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 943 time: 0.0817959 May 24 10:04:44.218481 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1299421f-c08b-437e-8fe4-d6bfc5572086 req-8c9defa4-a34e-4529-a96d-ca97663c3237 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0455589 May 24 10:04:44.416585 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:04:44.559658 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:44.573698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4', 'network_id': '2852c3dc-6508-45a7-9576-a36787475d18', 'port_id': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4', 'mac_address': 'fa:16:3e:e6:b3:9b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 63, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '9ee9ac65-ecfe-47d4-bbac-4561dabf43e6', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:04:44.635769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:44.654985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:04:44.671263 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:44.784970 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:44.788836 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:44.857733 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:44.867244 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.867736 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.868145 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.868546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.868941 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.869423 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.869814 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.870183 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.870482 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.870810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.871137 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.871461 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:44.877566 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:44.884843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57270) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:45.022109 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.037005 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:45.041780 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Usage tracker for resource:security_group_rule and tenant:3819d59662564c98bedb9eb6e2687ba2 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:45.044814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:45.050828 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:45.062581 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['086b9a1c-9348-4b08-b6f7-f934b5e8aaa4'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:04:45.135948 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.207759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.216703 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-aa008dec-8109-45fa-9a42-2ef67b47f126 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:45.258080 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=3819d59662564c98bedb9eb6e2687ba2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3719187 May 24 10:04:45.259583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f71493c9-009b-4b76-8976-f975cf8d1034,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:45.261388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57278) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:45.263255 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:45.398970 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.438311 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.488320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0572a15-dac6-4b70-b859-c2295656a9d1 req-5e1c39f9-e762-4e27-b764-e492cef130da tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:45.489994 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d0572a15-dac6-4b70-b859-c2295656a9d1 req-5e1c39f9-e762-4e27-b764-e492cef130da tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 944 time: 0.1558790 May 24 10:04:45.546334 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d0572a15-dac6-4b70-b859-c2295656a9d1 req-8addd0d7-dfc8-40f7-aab4-cec38ea253ba tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0474865 May 24 10:04:45.631035 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.636789 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2d4c5e43-2862-4e56-8052-a6a054490d8f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:45.651880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f71493c9-009b-4b76-8976-f975cf8d1034,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:45.655357 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:45.675465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:45.681513 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/f71493c9-009b-4b76-8976-f975cf8d1034 HTTP/1.1" status: 204 len: 168 time: 0.4135849 May 24 10:04:46.136823 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67244f43-10bb-44a3-8bcb-83139af11e75 req-9099c7ca-421d-4a57-81ac-d6c20e732daa tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:46.139009 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-67244f43-10bb-44a3-8bcb-83139af11e75 req-9099c7ca-421d-4a57-81ac-d6c20e732daa tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0880482 May 24 10:04:46.192387 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-67244f43-10bb-44a3-8bcb-83139af11e75 req-b2610ca1-4fd0-49d4-aaba-5195a1aea0fb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0475743 May 24 10:04:46.204543 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78ec617a-84b2-49f4-96d6-c290d30d555f req-6ddba469-83e7-4fa2-83f9-20856d425a21 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:46.206132 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78ec617a-84b2-49f4-96d6-c290d30d555f req-6ddba469-83e7-4fa2-83f9-20856d425a21 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0878086 May 24 10:04:46.254813 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-78ec617a-84b2-49f4-96d6-c290d30d555f req-40599cac-1eec-4978-82ba-80c9905cece5 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422163 May 24 10:04:46.429158 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:04:46.573611 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:04:46.581448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:46.582366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:46.707460 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:46.737568 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:04:46.754885 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-947744e4-9e11-42a6-a299-293e4a928e06 req-0930119b-f42a-4ca9-be77-08216fd70c5b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:46.756727 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-947744e4-9e11-42a6-a299-293e4a928e06 req-0930119b-f42a-4ca9-be77-08216fd70c5b tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 944 time: 0.0920930 May 24 10:04:46.757798 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:46.774544 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57318) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:46.796766 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4', 'code': 200} May 24 10:04:46.810023 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-947744e4-9e11-42a6-a299-293e4a928e06 req-9b132e69-fb0c-4978-ad18-9eaf16e0d47f tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0464427 May 24 10:04:46.830374 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:46.832225 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:46.900518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-1131347140-network', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:46.905530 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:46.907424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:46.909603 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Usage tracker for resource:network and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:46.910523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:46.912759 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:46.920831 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:47.007700 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Made reservation on behalf of 4b47c7cc67c545eab92ea86cde1abac7 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:04:47.010050 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.023194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.038306 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:47.042471 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:47.046383 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:47.051953 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:47.144593 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.173675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-917437b1-c128-4dab-b116-4226db96a690 req-f74d8e5c-80db-41f0-a9ec-3a61b1fe4135 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:47.175177 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-917437b1-c128-4dab-b116-4226db96a690 req-f74d8e5c-80db-41f0-a9ec-3a61b1fe4135 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 1098 time: 0.0870073 May 24 10:04:47.207521 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.211872 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2d9118ea-60df-4826-b3ee-990468b8402b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:47.233670 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-917437b1-c128-4dab-b116-4226db96a690 req-232ec573-c3d9-4479-a1fa-78103013f193 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0417941 May 24 10:04:47.241628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 82} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:04:47.256320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e83316d5-9d97-4e86-bce6-f8bde9c85f77,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:47.259119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.269742 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Added segment d0cd8b26-954b-4e20-bc84-0cd77d9d3588 of type vlan for network d34a1523-ea50-45ad-a897-459ffff7af90 May 24 10:04:47.269742 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:47.289844 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.398595 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c769a8b2-38d9-45d1-8591-241f8ae4b043 req-eb3e6246-f982-47f5-9c4d-96ae0986c050 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:47.399982 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c769a8b2-38d9-45d1-8591-241f8ae4b043 req-eb3e6246-f982-47f5-9c4d-96ae0986c050 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.0876472 May 24 10:04:47.419573 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.423201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:47.423768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:47.431008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:47.443589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b805194-1c4a-4317-abd1-345a7126b8d6 req-487e96d6-9263-414d-8380-92423faeec0b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:47.445082 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9b805194-1c4a-4317-abd1-345a7126b8d6 req-487e96d6-9263-414d-8380-92423faeec0b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0820866 May 24 10:04:47.491360 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9b805194-1c4a-4317-abd1-345a7126b8d6 req-0d9f6e05-9637-49ac-b32a-e8d54eebbcd7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404487 May 24 10:04:47.495363 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c769a8b2-38d9-45d1-8591-241f8ae4b043 req-b7dece55-a623-46c9-af97-91634f3e4020 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0890846 May 24 10:04:47.601747 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.8255978 May 24 10:04:47.604743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57328) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:47.610851 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Pushing event updated for resources: {'Network': ['ID=d34a1523-ea50-45ad-a897-459ffff7af90,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:47.611361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Request body: {'subnet': {'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-1131347140-subnet', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:47.614063 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.183s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:47.621586 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:04:47.623700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:47.628100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Usage tracker for resource:subnet and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:47.631172 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:47.636840 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:47.733581 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Made reservation on behalf of 4b47c7cc67c545eab92ea86cde1abac7 for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:04:47.735908 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-128981363'] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:48.150198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-c4bf153d-2ce4-4d2f-bb01-7ac73134e84e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:48.151721 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-c4bf153d-2ce4-4d2f-bb01-7ac73134e84e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.2608490 May 24 10:04:48.159543 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61984477-d434-44f9-82ba-e276c3d8fff0 req-4331143d-0c7a-4759-b6f4-056b301d52c8 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:48.160854 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-61984477-d434-44f9-82ba-e276c3d8fff0 req-4331143d-0c7a-4759-b6f4-056b301d52c8 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 945 time: 0.1589212 May 24 10:04:48.341645 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-61984477-d434-44f9-82ba-e276c3d8fff0 req-2e5006d2-ae2a-4ee8-b1da-79a2965a3b42 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1429698 May 24 10:04:48.432838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Deleting port 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:04:48.433114 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:48.620192 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] For port 6fec3696-3d1d-4762-9674-fe87fd37e211, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=6fec3696-3d1d-4762-9674-fe87fd37e211,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:48.620581 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:48.701172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70538099-2071-47b8-bfbb-8a7f0ba3bbee req-084f4308-f4e2-4746-b5bc-69f8b62ab772 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:48.702547 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-70538099-2071-47b8-bfbb-8a7f0ba3bbee req-084f4308-f4e2-4746-b5bc-69f8b62ab772 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0847027 May 24 10:04:48.747732 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-129003893', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-128981349'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:48.752129 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-70538099-2071-47b8-bfbb-8a7f0ba3bbee req-1591d92c-41cb-4fc1-a4d2-264600082e15 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434704 May 24 10:04:48.767361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:48.933727 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Pushing event updated for resources: {'Subnet': ['ID=a1ecfc88-eaf2-426e-bb90-f6a474fb5991,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:48.941812 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.174s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:48.952320 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Checking segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:48.952591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Network hostable DHCP agents. Network: d34a1523-ea50-45ad-a897-459ffff7af90, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:04:48.979845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-76bbb125-844c-47fb-8c20-3027c45bd964 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:48.981306 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-76bbb125-844c-47fb-8c20-3027c45bd964 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 945 time: 0.0851600 May 24 10:04:49.053275 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a98fe3e-6c95-4d06-b625-43eba6c93dd9 req-6c0f7c2b-3195-486c-9ceb-5987e5d85e43 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:49.054741 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3a98fe3e-6c95-4d06-b625-43eba6c93dd9 req-6c0f7c2b-3195-486c-9ceb-5987e5d85e43 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 947 time: 0.4366071 May 24 10:04:49.109833 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.110193 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.110509 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Floating IP 29cfb1c8-436d-4666-a731-b479717e97c6 disassociated. External IP: 172.27.0.38, port: 6fec3696-3d1d-4762-9674-fe87fd37e211. May 24 10:04:49.110851 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Calling delete_port for 6fec3696-3d1d-4762-9674-fe87fd37e211 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:04:49.119921 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Deleting port c7f85b81-de77-4ee2-8223-a1fb751791f5 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:04:49.120261 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.124827 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3a98fe3e-6c95-4d06-b625-43eba6c93dd9 req-05b284c2-141e-44c7-8172-6d3e7da3225b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=d4764188-82ae-444d-ae05-a4e09e8d56dd&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0627387 May 24 10:04:49.161921 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Network d34a1523-ea50-45ad-a897-459ffff7af90 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:04:49.241764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, d34a1523-ea50-45ad-a897-459ffff7af90) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:49.252555 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e61febb2-e114-4319-ac1d-e1901f20a0e9 None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:04:49.257299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:49.258179 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.259101 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] For port c7f85b81-de77-4ee2-8223-a1fb751791f5, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=c7f85b81-de77-4ee2-8223-a1fb751791f5,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:49.259462 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.293779 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.304172 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-30c06f6b-978b-4ec6-b030-17c8ad2f30bf tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:49.352949 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'Port': ['ID=6fec3696-3d1d-4762-9674-fe87fd37e211,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:49.360863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:49.398030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Calling delete_port for c7f85b81-de77-4ee2-8223-a1fb751791f5 owned by compute:nova {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:04:49.426087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-58a9309b-ea77-49bf-aaf4-0cf815420223 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 1.8198788 May 24 10:04:49.426475 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57340) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:49.435231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:49.443141 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:49.450668 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Usage tracker for resource:router and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:49.455020 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:49.463205 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:49.522273 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-d34a1523-ea50-45ad-a897-459ffff7af90', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:04:49.523270 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.578898 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.579907 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.635687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca044139-2057-4247-afde-67df86e8fd89 req-a049f800-8740-4c41-80f7-6cfa4cbc31fb tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:49.637021 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ca044139-2057-4247-afde-67df86e8fd89 req-a049f800-8740-4c41-80f7-6cfa4cbc31fb tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 945 time: 0.1553879 May 24 10:04:49.687167 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ca044139-2057-4247-afde-67df86e8fd89 req-8f857d8f-4947-4539-81a9-a5ae2d0ac8a2 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427682 May 24 10:04:49.696208 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:04:49.707982 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Allocated IP 172.20.0.2 (d34a1523-ea50-45ad-a897-459ffff7af90/a1ecfc88-eaf2-426e-bb90-f6a474fb5991/4bb8a808-7920-4267-8cbc-98c3ce9c0bab) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:04:49.720420 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.723625 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.729740 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:49.793407 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Pushing event deleted for resources: {'Port': ['ID=c7f85b81-de77-4ee2-8223-a1fb751791f5,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:49.797008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:49.900977 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] No DVR routers for this DVR port c7f85b81-de77-4ee2-8223-a1fb751791f5 on host hv-cinder-79123 {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:04:49.905823 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.911560 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'name': 'network-vif-deleted', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:04:49.932857 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:49.943175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Transition to ACTIVE for port object 4bb8a808-7920-4267-8cbc-98c3ce9c0bab will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:49.963631 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Transition to ACTIVE for port object 4bb8a808-7920-4267-8cbc-98c3ce9c0bab will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:49.979271 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['de553ab6-e238-403d-827d-b143492c1232', 'c7f85b81-de77-4ee2-8223-a1fb751791f5'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:04:50.049542 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:04:50.065829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.069846 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:04:50.076394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:50.077257 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e77cecd2-a366-4044-a748-7faaa37830a4 req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "DELETE /v2.0/ports/c7f85b81-de77-4ee2-8223-a1fb751791f5 HTTP/1.1" status: 204 len: 173 time: 1.0908313 May 24 10:04:50.088761 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-7b7366e5-41b8-45e2-b1d8-2457400448d1 req-28b6d2c0-8c76-4bfd-9223-01a4c391c8c7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:50.095353 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:50.098522 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:50.165237 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:50.178033 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:50.184919 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:04:50.194336 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Pushing event updated for resources: {'Port': ['ID=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:50.206700 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:50.247180 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] No DVR routers for this DVR port 6fec3696-3d1d-4762-9674-fe87fd37e211 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:04:50.253811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.258960 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0e4d8098-4250-4dc1-83e8-e4aed6016267 req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "DELETE /v2.0/ports/6fec3696-3d1d-4762-9674-fe87fd37e211 HTTP/1.1" status: 204 len: 173 time: 2.0938294 May 24 10:04:50.259610 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-deleted', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:04:50.300229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-7b7366e5-41b8-45e2-b1d8-2457400448d1 req-28b6d2c0-8c76-4bfd-9223-01a4c391c8c7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:50.304111 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.305661 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7b7366e5-41b8-45e2-b1d8-2457400448d1 req-28b6d2c0-8c76-4bfd-9223-01a4c391c8c7 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.2855132 May 24 10:04:50.337756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.422233 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:04:50.426657 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e740327b-669e-49e2-9b7c-1f72d272e5e4', 'name': 'network-vif-deleted', 'tag': 'c7f85b81-de77-4ee2-8223-a1fb751791f5', 'status': 'completed', 'code': 200} May 24 10:04:50.431371 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:04:50.470695 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 6fec3696-3d1d-4762-9674-fe87fd37e211 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:04:50.471044 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 6fec3696-3d1d-4762-9674-fe87fd37e211 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:04:50.590281 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7becbb7a-76e6-48f0-8280-c222b40333bb req-5a0b02b0-8635-41f8-b772-d44168d4c8a7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.0683794 May 24 10:04:50.760698 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '02d59782-db8b-4885-a4ec-a57499c07fe1', 'name': 'network-vif-deleted', 'tag': '6fec3696-3d1d-4762-9674-fe87fd37e211', 'status': 'completed', 'code': 200} May 24 10:04:50.780378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Attempting to bind port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:04:50.781214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Attempting to bind port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:04:50.781801 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Attempting to bind port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab on network d34a1523-ea50-45ad-a897-459ffff7af90 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:04:50.789921 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:04:50.790615 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Checking segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:50.792756 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.792756 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Bound using segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:04:50.796411 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:50.798053 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Bound port: 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:04:50.801484 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:04:50.801630 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.813514 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:50.814753 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:50.922126 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] For port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:04:50.927366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] For port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:04:50.994067 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:50.999964 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:51.036362 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:51.059426 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:04:51.071160 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:51.137066 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Pushing event updated for resources: {'Port': ['ID=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:51.148511 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:51.260866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Allocated IP 172.27.0.116 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/8c72ce53-67f4-42a6-beef-bb11cea7beb2) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:04:51.498345 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:51.571668 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:51.576614 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-62ad46ac-1d6e-4751-855a-190e82d9103d None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:04:51.653186 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5869bd25-0825-45d3-947f-a30b4e27e7c6 req-e04b5bc1-5d67-4beb-a687-9f43e9880d64 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=e740327b-669e-49e2-9b7c-1f72d272e5e4 HTTP/1.1" status: 200 len: 210 time: 0.0829422 May 24 10:04:51.819711 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3e0c6e6b-5f78-4da0-a6d9-67449043da69 req-f7c61047-5115-40d5-8c0a-341672e4a560 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:51.877077 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:51.882069 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:51.916326 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3e0c6e6b-5f78-4da0-a6d9-67449043da69 req-f7c61047-5115-40d5-8c0a-341672e4a560 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:51.917669 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3e0c6e6b-5f78-4da0-a6d9-67449043da69 req-f7c61047-5115-40d5-8c0a-341672e4a560 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=02d59782-db8b-4885-a4ec-a57499c07fe1 HTTP/1.1" status: 200 len: 210 time: 0.1399267 May 24 10:04:51.957832 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:51.959044 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:51.960869 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Pushing event updated for resources: {'Port': ['ID=8c72ce53-67f4-42a6-beef-bb11cea7beb2,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:51.964566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:04:52.014984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:52.419159 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fb9ea8e9-8c99-4df0-9f6a-c9c1445d9559 None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:04:52.451202 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac7b627d-c708-445d-8f6c-fcd600df9e0e None None] New status for floating IP 29cfb1c8-436d-4666-a731-b479717e97c6: DOWN {{(pid=109114) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:04:52.497640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:52.516217 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c7f85b81-de77-4ee2-8223-a1fb751791f5 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:04:52.559764 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4bb8a808-7920-4267-8cbc-98c3ce9c0bab up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:04:52.562724 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-0f424c3e-04e1-43d3-aa13-fe53082db031 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1044295 May 24 10:04:52.672807 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: c7f85b81-de77-4ee2-8223-a1fb751791f5 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:04:52.673179 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device c7f85b81-de77-4ee2-8223-a1fb751791f5 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:04:52.675259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-8ca3ab11-a5ab-4d17-8171-bc452d87b052 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:52.677198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-8ca3ab11-a5ab-4d17-8171-bc452d87b052 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 1098 time: 0.0881736 May 24 10:04:52.837343 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-508ad34f-a931-4703-b20b-f7de93f4f5ae None None] Provisioning for port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:04:52.880233 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-9cb6d453-d695-4375-8a23-c63369d0ac32 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:52.881948 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-9cb6d453-d695-4375-8a23-c63369d0ac32 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.1797431 May 24 10:04:53.041839 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.154943 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-9c60717e-e3bd-45a4-b94b-785dc1c84b2f service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 200 len: 1000 time: 0.2646344 May 24 10:04:53.175330 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:04:53.191022 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:53.191399 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.221847 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-3585b1bd-7da5-4c35-9007-b1898322b725 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:53.223434 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-3585b1bd-7da5-4c35-9007-b1898322b725 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.0620878 May 24 10:04:53.293271 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57378) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:53.330239 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.341567 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-abdd7de6-4a4d-4b29-bc84-97522cc3c8cf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:53.343293 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-abdd7de6-4a4d-4b29-bc84-97522cc3c8cf service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1095252 May 24 10:04:53.387145 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:53.404881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.443666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.470848 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.472736 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ecab8ca2-1e94-450f-ac40-66c87fdc78a7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:53.545135 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Router 7ead47d0-d22c-480f-8edb-a291a66ba849 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:04:53.574194 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:53.582431 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-76da64f3-12ac-4aaa-ab0b-69ad7adf5169 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2242737 May 24 10:04:53.585934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:53.601410 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-b26d4b95-7d6e-4d35-b02a-61fdd5411faf None None] Provisioning complete for port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:53.601791 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b26d4b95-7d6e-4d35-b02a-61fdd5411faf None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.666123 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.673225 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e55a9ef9-2bf9-4072-a4f8-88ef116d7186 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:53.699511 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=d4764188-82ae-444d-ae05-a4e09e8d56dd,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:53.705104 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:53.740698 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f79b2be7-065d-4754-ab82-8ee6a39e7a40 req-ad36830b-d335-42e1-b924-16e82b11d62d service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1493535 May 24 10:04:53.814647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.815741 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "DELETE /v2.0/security-groups/d4764188-82ae-444d-ae05-a4e09e8d56dd HTTP/1.1" status: 204 len: 168 time: 0.5215523 May 24 10:04:53.818745 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57386) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:53.824658 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:04:53.834578 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Sync routers for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:04:53.835087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.api.v2.resource [None req-e5a78561-96f8-4e66-9824-eb695f6bdab7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] show failed (client error): The resource could not be found. May 24 10:04:53.838918 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e5a78561-96f8-4e66-9824-eb695f6bdab7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups/d4764188-82ae-444d-ae05-a4e09e8d56dd HTTP/1.1" status: 404 len: 347 time: 0.0162647 May 24 10:04:53.839061 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57390) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:53.846589 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:53.847209 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:53.848506 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf24ed2b-b70d-490a-9af8-c0abc419e4dd req-cc6bc28d-310b-474b-a008-217b5df88f13 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:53.850178 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bf24ed2b-b70d-490a-9af8-c0abc419e4dd req-cc6bc28d-310b-474b-a008-217b5df88f13 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.2390556 May 24 10:04:53.900282 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bf24ed2b-b70d-490a-9af8-c0abc419e4dd req-b358e826-7d88-49b9-87ef-980debc8310c tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434089 May 24 10:04:53.910448 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:53.912483 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 927 time: 4.4850507 May 24 10:04:53.915153 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57394) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:54.030779 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:04:54.070500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Deleting port 43451a90-88e8-4a57-b11e-8d47a160d020 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:04:54.071069 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.334640 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.335068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.335318 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:52Z', 'revision_number': 3, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:51Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:54.340018 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Checking router: 7ead47d0-d22c-480f-8edb-a291a66ba849 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:04:54.340447 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Updating router 7ead47d0-d22c-480f-8edb-a291a66ba849 port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:04:54.400399 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.444146 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4bb8a808-7920-4267-8cbc-98c3ce9c0bab up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:04:54.466685 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Calling delete_port for 43451a90-88e8-4a57-b11e-8d47a160d020 owned by network:floatingip {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:04:54.612556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:54.615612 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.642689 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.741575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.755783 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:54.835533 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'Port': ['ID=43451a90-88e8-4a57-b11e-8d47a160d020,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:54.840290 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.852945 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Transition to ACTIVE for port object 8c72ce53-67f4-42a6-beef-bb11cea7beb2 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:54.866359 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:54.894088 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.905577 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.910460 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:54.956931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Allocated IP 172.20.0.1 (d34a1523-ea50-45ad-a897-459ffff7af90/a1ecfc88-eaf2-426e-bb90-f6a474fb5991/eca6df41-ad3d-4f1a-9a5c-cd20b4f97938) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:04:54.974151 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:54.975282 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "DELETE /v2.0/floatingips/29cfb1c8-436d-4666-a731-b479717e97c6 HTTP/1.1" status: 204 len: 168 time: 1.1354377 May 24 10:04:55.000092 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Attempting to bind port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:04:55.000296 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Attempting to bind port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:04:55.000921 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Attempting to bind port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:04:55.007789 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Pushing event updated for resources: {'Port': ['ID=8c72ce53-67f4-42a6-beef-bb11cea7beb2,revision_number=2']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:55.010542 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:04:55.010723 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:55.011050 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.011675 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:04:55.011807 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Bound port: 8c72ce53-67f4-42a6-beef-bb11cea7beb2, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:04:55.012165 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.123751 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.124114 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.124453 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.124777 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.125120 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.125446 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.125765 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.126079 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.126413 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.126733 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.127049 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.127358 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:04:55.131587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3988ae5e-5953-472b-9e4d-48ebfa26f72c req-8ceae93f-796d-457e-aacc-edb7bb63a65a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:55.132749 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:55.135623 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3988ae5e-5953-472b-9e4d-48ebfa26f72c req-8ceae93f-796d-457e-aacc-edb7bb63a65a tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1021578 May 24 10:04:55.140565 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] For port 8c72ce53-67f4-42a6-beef-bb11cea7beb2, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:04:55.145946 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] For port 8c72ce53-67f4-42a6-beef-bb11cea7beb2, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8c72ce53-67f4-42a6-beef-bb11cea7beb2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:04:55.162570 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.178649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Transition to ACTIVE for port object eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:55.185146 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3988ae5e-5953-472b-9e4d-48ebfa26f72c req-88826796-5844-4a26-8c65-b326adcd0160 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0406916 May 24 10:04:55.194285 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.202640 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:55.236727 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.241751 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:55.273813 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:55.274446 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:55.303804 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Network d34a1523-ea50-45ad-a897-459ffff7af90 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:55.306644 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:55.310888 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:04:55.311353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.324640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Pushing event updated for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:55.334882 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:55.358211 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Pushing event updated for resources: {'Port': ['ID=8c72ce53-67f4-42a6-beef-bb11cea7beb2,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:55.362532 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7c98b6e3-65c3-492b-90f9-6f1e53787907 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:55.466739 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:04:55.532389 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57404) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:55.537723 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1067280220', 'description': 'tempest-description--845075305'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:55.544274 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:55.548773 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:55.551729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:55.558141 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:55.575261 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.671259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.693120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:55.698549 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:55.701961 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:55.708517 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:55.710084 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-21f7c0a2-d774-458c-8223-4ecfa0d71465 None None] Provisioning for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:04:55.715889 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-21f7c0a2-d774-458c-8223-4ecfa0d71465 None None] Provisioning complete for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:55.716341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-21f7c0a2-d774-458c-8223-4ecfa0d71465 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.768803 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:52Z', 'revision_number': 3, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:55.773616 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7cf276eb-7208-4023-9f0c-f4fdf2a09db0 None None] The sync data for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi is: [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:52Z', 'revision_number': 3, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:04:55.793168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-21f7c0a2-d774-458c-8223-4ecfa0d71465 None None] Port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:04:55.825843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.838195 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.922060 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.925957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c7fbff8d-8ac5-4957-bdfb-44e4157fbc68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:55.938820 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.939594 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:55.943345 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7b08ebaa-b150-4280-95b2-1f78fff19deb tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:55.975918 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroup': ['ID=b0a9d06a-7b60-46b7-9ded-3120b0dc1680,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:55.979197 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:56.060485 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Pushing event updated for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:56.060623 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.064745 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:56.068111 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.5320001 May 24 10:04:56.069221 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57410) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:56.075218 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group_rule': {'security_group_id': 'b0a9d06a-7b60-46b7-9ded-3120b0dc1680', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:56.081749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:56.087395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:56.090612 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:56.097778 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:56.165760 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Network d34a1523-ea50-45ad-a897-459ffff7af90 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:56.167305 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:56.185114 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.275229 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.391916 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.398005 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5d556fcf-79c7-4450-b88b-f62066f55b78 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:56.425687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Router 7ead47d0-d22c-480f-8edb-a291a66ba849 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:04:56.434449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-70daddbf-fc44-4554-9c31-a54edb68eb65 None None] Provisioning complete for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:56.435042 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-70daddbf-fc44-4554-9c31-a54edb68eb65 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.436067 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=229f4116-7ae2-4ff5-866e-f0b7b1e1c8ea,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:56.443285 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:56.470566 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d7b10cf8-6e52-4f0e-aad2-97b5e68a4068 req-b0289f82-e840-40f6-9836-b5c961649ea8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:56.478140 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d7b10cf8-6e52-4f0e-aad2-97b5e68a4068 req-b0289f82-e840-40f6-9836-b5c961649ea8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1385550 May 24 10:04:56.523026 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8c72ce53-67f4-42a6-beef-bb11cea7beb2 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:04:56.552416 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d7b10cf8-6e52-4f0e-aad2-97b5e68a4068 req-5cba9cca-e4f5-492f-ac8c-e6b972216b9b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0740762 May 24 10:04:56.559552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.561128 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.4906902 May 24 10:04:56.563514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57416) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:56.568973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'security_group_rule': {'security_group_id': 'b0a9d06a-7b60-46b7-9ded-3120b0dc1680', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:56.569194 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-70daddbf-fc44-4554-9c31-a54edb68eb65 None None] Port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:04:56.584081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:56.590895 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:security_group_rule and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:56.595633 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:56.603596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:56.647491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:04:56.657743 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] 10.222.0.67 "PUT /v2.0/routers/7ead47d0-d22c-480f-8edb-a291a66ba849/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.7415011 May 24 10:04:56.696214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:04:56.698629 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:56.699039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.724853 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.803726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.844675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.867428 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 8c72ce53-67f4-42a6-beef-bb11cea7beb2, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8c72ce53-67f4-42a6-beef-bb11cea7beb2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:56.881090 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.889524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:56.908016 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=05defd11-dfd0-4519-a3dd-253b4e59ecd8,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:56.911402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:56.995905 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:56.999493 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:57.018991 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.020431 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4556923 May 24 10:04:57.023284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57428) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:57.028909 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:57.036390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:57.040471 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Usage tracker for resource:floatingip and tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:57.043695 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:57.049506 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:57.069564 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=8c72ce53-67f4-42a6-beef-bb11cea7beb2,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:57.072441 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:57.162104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.305344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.428385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57440) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:57.434607 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-2111530498-network', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:57.439985 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:57.444136 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Usage tracker for resource:network and tenant:f7673d41f8314855baa9c60fd5e2e1ba is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:57.447061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:57.450472 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Allocated IP 172.27.0.60 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/842d1931-9818-44e9-b880-ba879c97aaa3) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:04:57.452812 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:57.479565 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Sync routers for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:04:57.489239 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:57.491676 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:57.545369 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Made reservation on behalf of f7673d41f8314855baa9c60fd5e2e1ba for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:04:57.548017 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.560473 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.576288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:57.580789 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Usage tracker for resource:security_group_rule and tenant:f7673d41f8314855baa9c60fd5e2e1ba is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:57.584443 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:57.592812 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:57.610587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.686070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:04:57.713785 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.724653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:57.743415 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.776103 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-376a3264-0222-4b97-8297-b5ce46ad60f1 req-432cbefc-0e0b-44a6-80a5-10b8922ef33b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:57.777898 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-376a3264-0222-4b97-8297-b5ce46ad60f1 req-432cbefc-0e0b-44a6-80a5-10b8922ef33b tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.0995803 May 24 10:04:57.785037 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:04:57.788663 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:57.792655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:04:57.826800 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-376a3264-0222-4b97-8297-b5ce46ad60f1 req-f4e26e11-047a-4cbc-bdeb-b3db3c5903c8 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420794 May 24 10:04:57.837677 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.851061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3c250553-3597-4f88-a631-cbf96a7f82fa admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:57.851281 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=842d1931-9818-44e9-b880-ba879c97aaa3,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:57.852464 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:57.878548 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 97} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:04:57.888305 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f369c076-92b5-438f-b571-9bbb565f2aae,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:57.892063 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:57.895089 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.895515 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Added segment 62003036-d603-4e95-8128-f4d87aace384 of type vlan for network a5c48f0c-499f-4abd-b04c-ed57773dbaef May 24 10:04:57.923575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:57.931402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.056856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.067023 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 2, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:58.071320 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Checking router: 7ead47d0-d22c-480f-8edb-a291a66ba849 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:04:58.071683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Updating router 7ead47d0-d22c-480f-8edb-a291a66ba849 port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:04:58.099732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.101141 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:58.102144 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.109968 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:58.137908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.224845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Pushing event updated for resources: {'Network': ['ID=a5c48f0c-499f-4abd-b04c-ed57773dbaef,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:58.228763 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:58.236309 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.8065035 May 24 10:04:58.239107 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57448) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:58.244638 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Request body: {'subnet': {'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-2111530498-subnet', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:58.249768 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:04:58.250659 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:58.254609 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Usage tracker for resource:subnet and tenant:f7673d41f8314855baa9c60fd5e2e1ba is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:58.257463 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:58.264539 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:58.349817 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Made reservation on behalf of f7673d41f8314855baa9c60fd5e2e1ba for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:04:58.352174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Notify callbacks [] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.363103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.437114 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.443050 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f2a5275b-21a7-4766-92de-242e644a4c03 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:58.469197 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8c72ce53-67f4-42a6-beef-bb11cea7beb2 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:04:58.572793 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=842d1931-9818-44e9-b880-ba879c97aaa3,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:58.581190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:58.587152 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.632730 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:04:58.633125 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.682219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.683598 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.6593668 May 24 10:04:58.772214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.818058 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Transition to ACTIVE for port object eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:04:58.900981 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:58.905846 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-89d69f5c-9d75-4de1-9c7a-4ee922069b4f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:58.955202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-39564b0e-3af7-4ac5-9f42-9e92a625d9e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:59.007276 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.012299 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:59.036318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Pushing event updated for resources: {'Subnet': ['ID=ffc35f12-deef-45c7-9064-c4d2fba75cd3,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:59.039792 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Checking segment: {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:59.040144 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Network hostable DHCP agents. Network: a5c48f0c-499f-4abd-b04c-ed57773dbaef, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:04:59.049013 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:59.083977 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc26a2d3-10ee-4f84-93b9-b0e12a7f263e req-8c7f34f3-d926-4872-84d0-6769b3359653 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:59.085581 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bc26a2d3-10ee-4f84-93b9-b0e12a7f263e req-8c7f34f3-d926-4872-84d0-6769b3359653 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1228559 May 24 10:04:59.122334 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-39564b0e-3af7-4ac5-9f42-9e92a625d9e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:59.123495 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-39564b0e-3af7-4ac5-9f42-9e92a625d9e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-securitygroup--1067280220&fields=id&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 267 time: 0.2110579 May 24 10:04:59.168700 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bc26a2d3-10ee-4f84-93b9-b0e12a7f263e req-78923e37-a2e4-410e-bb8f-10f069530b2f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0745518 May 24 10:04:59.186468 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:04:59.267264 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Pushing event updated for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:59.285752 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.273s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:04:59.289005 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Attempting to bind port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:04:59.289544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Attempting to bind port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:04:59.290088 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Attempting to bind port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 on network d34a1523-ea50-45ad-a897-459ffff7af90 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:04:59.298713 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, a5c48f0c-499f-4abd-b04c-ed57773dbaef) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:04:59.301799 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:04:59.302635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Checking segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:04:59.303604 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.304251 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Bound using segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:04:59.305034 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Bound port: eca6df41-ad3d-4f1a-9a5c-cd20b4f97938, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:04:59.306521 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.309568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:59.311785 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.333543 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d0f5ada7-62d4-487c-ae62-3c890b170613 None None] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:04:59.353865 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-c946b42c-f1e8-49af-9f8c-862e8772fa18 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:59.361777 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-c946b42c-f1e8-49af-9f8c-862e8772fa18 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 817 time: 0.2288530 May 24 10:04:59.372594 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-aa18467d-445c-46e8-9fee-1a9262bd99ac tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/quotas/c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 375 time: 0.0096776 May 24 10:04:59.451297 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 1.2081068 May 24 10:04:59.451530 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57470) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:04:59.459401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:04:59.464082 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:04:59.468866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Usage tracker for resource:router and tenant:f7673d41f8314855baa9c60fd5e2e1ba is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:04:59.472481 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:04:59.500567 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:04:59.527528 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] For port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:04:59.530135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3784e100-7058-4630-8405-c791de8f872c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:04:59.535668 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] For port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:04:59.545441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-4e413a6b-0f94-4bf4-bcc5-33a16c987d43 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:04:59.546800 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-4e413a6b-0f94-4bf4-bcc5-33a16c987d43 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1665933 May 24 10:04:59.601964 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.603107 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.626762 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.660117 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'fixed_ips': [{'subnet_id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:04:59.661265 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.700416 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Deleting port de553ab6-e238-403d-827d-b143492c1232 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:04:59.701316 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.730432 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.733232 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3784e100-7058-4630-8405-c791de8f872c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:04:59.734561 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3784e100-7058-4630-8405-c791de8f872c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&fields=id HTTP/1.1" status: 200 len: 348 time: 0.3560975 May 24 10:04:59.739095 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:04:59.907891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:04:59.908537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:04:59.949122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] For port de553ab6-e238-403d-827d-b143492c1232, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=de553ab6-e238-403d-827d-b143492c1232,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:04:59.949502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:04:59.968444 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Pushing event updated for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:04:59.979421 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.240s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:00.011065 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Allocated IP 172.20.0.2 (a5c48f0c-499f-4abd-b04c-ed57773dbaef/ffc35f12-deef-45c7-9064-c4d2fba75cd3/ad6248be-368d-44d8-9aaa-9ac7d6960750) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:00.078520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.102060 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Calling delete_port for de553ab6-e238-403d-827d-b143492c1232 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:00.116863 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-1ccf36af-c428-4ae0-8352-65068c5f724e service neutron] 10.222.0.67 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3249903 May 24 10:05:00.206522 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:00.276595 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-c703ec5d-6bf9-4018-a47c-4230c00893e2 service neutron] 10.222.0.67 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1566389 May 24 10:05:00.416693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53ebc4dc-9429-46ff-a7d5-dd90386b278f req-96c8b46a-5539-47cb-9c5c-87ea2d309b63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:00.418234 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53ebc4dc-9429-46ff-a7d5-dd90386b278f req-96c8b46a-5539-47cb-9c5c-87ea2d309b63 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 945 time: 0.1202924 May 24 10:05:00.447182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.453798 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-90ff0fd6-5af5-4da2-8466-7e86bde20b52 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:00.570328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Pushing event deleted for resources: {'Port': ['ID=de553ab6-e238-403d-827d-b143492c1232,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:00.580855 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:00.668523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.687744 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Transition to ACTIVE for port object ad6248be-368d-44d8-9aaa-9ac7d6960750 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:00.714775 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53ebc4dc-9429-46ff-a7d5-dd90386b278f req-05279dae-5955-4fde-a8ec-055903715c7d tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/security-groups?id=fe64f46b-8749-477e-bbac-58ca92a691c2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2462049 May 24 10:05:00.729614 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c27da7e6-e699-4672-9aa7-6b656141302c req-04736e8d-d6e2-4d40-8f41-38ff73e01ec2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.1223817 May 24 10:05:00.733519 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] No DVR routers for this DVR port de553ab6-e238-403d-827d-b143492c1232 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:05:00.736733 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.738662 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Transition to ACTIVE for port object ad6248be-368d-44d8-9aaa-9ac7d6960750 will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:00.748691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.749020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'name': 'network-vif-deleted', 'tag': 'de553ab6-e238-403d-827d-b143492c1232'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:00.779314 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:04:59Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:00.779488 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a8fc43c4-b5e4-4caa-b93e-30ec7f7fd783 None None] The sync data for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi is: [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:04:59Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:00.785562 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.806883 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5b161895-21f1-4c71-9f5f-4d521eaea323', 'name': 'network-vif-deleted', 'tag': 'de553ab6-e238-403d-827d-b143492c1232', 'status': 'completed', 'code': 200} May 24 10:05:00.858394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:00.861008 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:00.923037 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d885a30b-fc3c-4dde-955a-aa1e84933ca6 req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.95 "DELETE /v2.0/ports/de553ab6-e238-403d-827d-b143492c1232 HTTP/1.1" status: 204 len: 173 time: 1.3662941 May 24 10:05:00.933201 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:00.936203 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:00.940623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:00.942807 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Pushing event updated for resources: {'Port': ['ID=ad6248be-368d-44d8-9aaa-9ac7d6960750,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:00.946613 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:00.982839 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Allocated IP 172.27.0.145 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/092d3b12-88bd-4f00-a17e-3ebfeb696ec6) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:01.039733 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Attempting to bind port ad6248be-368d-44d8-9aaa-9ac7d6960750 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:01.040033 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Attempting to bind port ad6248be-368d-44d8-9aaa-9ac7d6960750 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:01.040415 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Attempting to bind port ad6248be-368d-44d8-9aaa-9ac7d6960750 on network a5c48f0c-499f-4abd-b04c-ed57773dbaef {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:01.047222 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:01.047580 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Checking segment: {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:01.048035 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.048872 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Bound using segment: {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:01.048995 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Bound port: ad6248be-368d-44d8-9aaa-9ac7d6960750, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:01.049398 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.150360 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.160532 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] For port ad6248be-368d-44d8-9aaa-9ac7d6960750, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:01.164467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] For port ad6248be-368d-44d8-9aaa-9ac7d6960750, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ad6248be-368d-44d8-9aaa-9ac7d6960750,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:01.265417 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.265973 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.271305 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:01.275698 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-880f5e3a-1e7e-422c-a05d-7a467f38efc0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:01.333825 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:01.337399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:01.341928 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:01.365010 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Pushing event updated for resources: {'Port': ['ID=ad6248be-368d-44d8-9aaa-9ac7d6960750,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:01.368497 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:01.417668 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Pushing event updated for resources: {'Port': ['ID=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:01.428116 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:01.907445 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-e7d5bd87-c362-45e3-bba5-0c8b4ecd4016 req-3db9bab3-ef17-4d74-8bcb-41a281a28154 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Synchronizing usage tracker for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:01.943335 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ee529a04-1acf-4fdd-8eeb-f79628db7c13 req-dfcd1180-f160-459d-bc6b-606ee5b4afdd tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.0482702 May 24 10:05:01.953326 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:01.988042 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-e7d5bd87-c362-45e3-bba5-0c8b4ecd4016 req-3db9bab3-ef17-4d74-8bcb-41a281a28154 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Unset dirty status for tenant:b08409afad374227ace1cdf31e4e9412 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:01.989574 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e7d5bd87-c362-45e3-bba5-0c8b4ecd4016 req-3db9bab3-ef17-4d74-8bcb-41a281a28154 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 210 time: 0.1317990 May 24 10:05:02.246961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4c75fa4d-0b66-4199-8be2-2788d08ab825 None None] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:05:02.294944 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:02.424737 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-1b2faee2-2d0b-4372-8ff3-846bd905ba61 None None] Provisioning for port ad6248be-368d-44d8-9aaa-9ac7d6960750 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:02.518979 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:02.636225 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Router 66fcf049-caec-4596-8b28-a1655969d1af is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:05:02.676203 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:02.681480 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:02.681847 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:02.817597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:02.822250 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:05:02.832274 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Sync routers for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:05:02.842108 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:02.842729 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:02.854524 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:02.916809 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:02.923025 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 927 time: 3.4667933 May 24 10:05:02.923988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57502) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:02.938601 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:02.939883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ad6248be-368d-44d8-9aaa-9ac7d6960750 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:02.965918 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:03.040197 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:03.090644 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['de553ab6-e238-403d-827d-b143492c1232'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:05:03.150532 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:03.168266 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.202s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:03.250364 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port ad6248be-368d-44d8-9aaa-9ac7d6960750 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:03.255068 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1536f4ec-d2fb-41fa-8939-35d13cd19ae4 req-5cde3da0-0ff6-4175-8a40-3ad7e60a79ea tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.1166925 May 24 10:05:03.265305 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port ad6248be-368d-44d8-9aaa-9ac7d6960750 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:03.265699 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.293654 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0c97414e-0fe7-4458-ba2f-046a17a9cbf4 req-c4d5cea6-96ce-4b2f-b371-57ce394fbb6f tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] 10.222.0.67 "GET /v2.0/ports?device_id=5b161895-21f1-4c71-9f5f-4d521eaea323 HTTP/1.1" status: 200 len: 210 time: 0.1743128 May 24 10:05:03.353011 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5313acc4-1bf0-4198-a1ee-8ea1f0326aeb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.358458 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:01Z', 'revision_number': 3, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:00Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:03.362550 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Checking router: 66fcf049-caec-4596-8b28-a1655969d1af for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:05:03.362902 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Updating router 66fcf049-caec-4596-8b28-a1655969d1af port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:05:03.420284 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.450339 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.487123 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:05:03.503255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port ad6248be-368d-44d8-9aaa-9ac7d6960750, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ad6248be-368d-44d8-9aaa-9ac7d6960750,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:03.596181 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.604292 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:03.676893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=ad6248be-368d-44d8-9aaa-9ac7d6960750,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:03.686906 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:03.802548 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.812256 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.824052 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Transition to ACTIVE for port object 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:03.892610 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.897599 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:03.980034 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Attempting to bind port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:03.980407 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Attempting to bind port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:03.981251 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Attempting to bind port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:03.984651 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Pushing event updated for resources: {'Port': ['ID=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,revision_number=2']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:03.989404 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:03.989785 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:03.990188 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:03.990577 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:03.991066 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Bound port: 092d3b12-88bd-4f00-a17e-3ebfeb696ec6, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:03.991422 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.038347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Allocated IP 172.20.0.1 (a5c48f0c-499f-4abd-b04c-ed57773dbaef/ffc35f12-deef-45c7-9064-c4d2fba75cd3/350a3cff-ca12-4254-a4cf-07a695eb5914) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:04.066640 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:04.098468 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] For port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:04.101878 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] For port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:04.190885 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.196038 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:04.214718 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.231064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Transition to ACTIVE for port object 350a3cff-ca12-4254-a4cf-07a695eb5914 will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:04.277867 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:04.278484 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:04:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:04.316632 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.320666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-70bab5cb-08aa-474e-b39c-dfd029f60c39 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:04.356039 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Pushing event updated for resources: {'Port': ['ID=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:04.361415 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:04.459513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57520) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:04.486897 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:04.516247 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:04.553817 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:04.556795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:04.560688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:04.561209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.571816 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Pushing event updated for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:04.608743 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.288s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:04.658019 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:04.658019 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.693393 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-09b68f89-8599-4dce-883e-53b104dcbac2 req-0babc50e-e72d-4b53-a208-35e57fa230c2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.1255250 May 24 10:05:04.837296 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ad6248be-368d-44d8-9aaa-9ac7d6960750 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:04.878005 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:01Z', 'revision_number': 3, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:04.879450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:04.884119 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e3fb205a-600d-4aa1-8546-0e413dcf516b None None] The sync data for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi is: [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:01Z', 'revision_number': 3, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:04.968886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port ad6248be-368d-44d8-9aaa-9ac7d6960750 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:04.969413 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.017445 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-d6c1040d-5036-4fd2-b34e-02856c514a16 None None] Provisioning for port 350a3cff-ca12-4254-a4cf-07a695eb5914 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:05.042161 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-d6c1040d-5036-4fd2-b34e-02856c514a16 None None] Provisioning complete for port 350a3cff-ca12-4254-a4cf-07a695eb5914 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:05.042659 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d6c1040d-5036-4fd2-b34e-02856c514a16 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.102107 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device de553ab6-e238-403d-827d-b143492c1232 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:05:05.141438 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: de553ab6-e238-403d-827d-b143492c1232 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:05:05.141917 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device de553ab6-e238-403d-827d-b143492c1232 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:05:05.154662 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.159155 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d6c1040d-5036-4fd2-b34e-02856c514a16 None None] Port 350a3cff-ca12-4254-a4cf-07a695eb5914 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:05:05.213673 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.216574 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.221818 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-28d10fcd-c57a-491d-a24d-a8730cf74eaa tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:05.279345 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-154d51ec-0bd0-4644-96b2-438d9a9f93b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:05.280984 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-154d51ec-0bd0-4644-96b2-438d9a9f93b3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 817 time: 0.3422291 May 24 10:05:05.414682 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Pushing event updated for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:05.422442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:05.473236 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.473772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:05.474637 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:05.685148 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Router 66fcf049-caec-4596-8b28-a1655969d1af has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:05:05.792502 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4df43e86-7fc0-4e19-9219-475702cb8f72 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.804606 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-32f5b947-4582-4782-92ff-f059730fe90f None None] Provisioning complete for port 350a3cff-ca12-4254-a4cf-07a695eb5914 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:05.805348 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-32f5b947-4582-4782-92ff-f059730fe90f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.881755 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-32f5b947-4582-4782-92ff-f059730fe90f None None] Port 350a3cff-ca12-4254-a4cf-07a695eb5914 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:05:05.905138 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:05.943136 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0800e599-9538-43a0-a66a-185205eecf53 req-1cfe21d4-980c-4bda-a442-50b47c47dc0a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.0539336 May 24 10:05:05.947214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:05:05.957500 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] 10.222.0.67 "PUT /v2.0/routers/66fcf049-caec-4596-8b28-a1655969d1af/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.0331843 May 24 10:05:06.026294 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Deleting port b154e178-27fc-4709-b3e0-94ee2928fd20 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:05:06.026721 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.150285 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] For port b154e178-27fc-4709-b3e0-94ee2928fd20, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b154e178-27fc-4709-b3e0-94ee2928fd20,segment=NetworkSegment(bb46500f-61f2-431d-97f0-2ba79eaa0fd2),segment_id=bb46500f-61f2-431d-97f0-2ba79eaa0fd2)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:06.150642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.368591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Calling delete_port for b154e178-27fc-4709-b3e0-94ee2928fd20 owned by network:router_interface {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:06.481375 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:06.624254 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:06.629452 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:06.629837 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.754115 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.767576 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.771898 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:06.772443 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:06.816189 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Pushing event deleted for resources: {'Port': ['ID=b154e178-27fc-4709-b3e0-94ee2928fd20,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:06.840895 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:06.859064 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:06.861119 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:06.930475 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:06.934366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:07.156891 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-da497db9-6397-4a9c-a0da-6074cdf37dad req-9c475fdf-2f58-4f72-9e29-93da5c856674 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.0617046 May 24 10:05:07.160018 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:07.215051 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:07.587050 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:05:07.731129 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Sync routers for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:05:07.746616 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:07.747250 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:07.940890 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:05:07.953923 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-498645a3-b544-467d-9840-3546986baa53 None None] Sync routers for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:05:07.955664 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:07.968404 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:07.969246 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:08.020736 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] 10.222.0.67 "PUT /v2.0/routers/8eac1314-02e8-4975-a4e9-f33c258a1bd0/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.5582311 May 24 10:05:08.023146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57564) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:08.209012 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:08.210779 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks [] for router, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:08.396660 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 4, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '350a3cff-ca12-4254-a4cf-07a695eb5914', 'name': '', 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'mac_address': 'fa:16:3e:5e:56:30', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_interface', 'standard_attr_id': 445, 'fixed_ips': [{'subnet_id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:03Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 2, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'subnets': [{'id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:08.403063 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Checking router: 66fcf049-caec-4596-8b28-a1655969d1af for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:05:08.404181 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Updating router 66fcf049-caec-4596-8b28-a1655969d1af port 350a3cff-ca12-4254-a4cf-07a695eb5914 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:05:08.437152 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-1bcead1c-2ad0-4611-8c73-7cd7a03684d0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 380 time: 0.1601655 May 24 10:05:08.449445 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'port': {'device_id': '812a56d8-d836-470a-b693-994765367aec', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'admin_state_up': True, 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'security_groups': ['b0a9d06a-7b60-46b7-9ded-3120b0dc1680']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:08.514009 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:08.553458 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8eb87f45-d010-435c-8ae1-718d4671195d req-0cf8b899-c60f-4ba3-b7e5-940a95fa5f65 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.1133189 May 24 10:05:08.645301 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:08.661376 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19'}], 'enable_snat': True}, 'gw_port_id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:05:06Z', 'revision_number': 5, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port': {'id': '97fff5ef-d87b-48e1-9d52-2ff0a4597cc4', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:c2:d9:ca', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 289, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.19', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:01:51Z', 'updated_at': '2021-05-24T10:01:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:08.666261 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-498645a3-b544-467d-9840-3546986baa53 None None] Checking router: 8eac1314-02e8-4975-a4e9-f33c258a1bd0 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:05:08.675280 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] Routers {'8eac1314-02e8-4975-a4e9-f33c258a1bd0'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:08.676205 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:08.697212 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:08.712922 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:08.713396 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:08.801556 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Made reservation on behalf of c42fc1bb2ae6477fa7b7c9261f2d50bb for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:05:08.805507 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:08.912192 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] Set router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 gateway port host: None {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:09.013861 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Deleting port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:05:09.014817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.065416 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.078046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Transition to ACTIVE for port object 350a3cff-ca12-4254-a4cf-07a695eb5914 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:09.108528 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:09.108912 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.146862 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-498645a3-b544-467d-9840-3546986baa53 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:05:08Z', 'revision_number': 6, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:09.151420 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-498645a3-b544-467d-9840-3546986baa53 None None] The sync data for ids ['8eac1314-02e8-4975-a4e9-f33c258a1bd0'] in dv-cinder-791239-2-iscsi is: [{'id': '8eac1314-02e8-4975-a4e9-f33c258a1bd0', 'name': 'tempest-ServerRescueNegativeTestJSON-786240911-router', 'tenant_id': 'b08409afad374227ace1cdf31e4e9412', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:01:50Z', 'updated_at': '2021-05-24T10:05:08Z', 'revision_number': 6, 'project_id': 'b08409afad374227ace1cdf31e4e9412', 'gw_port_host': None}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:09.170642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.177071 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:09.208587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Allocated IP 172.20.0.13 (c1fb75f5-deee-4236-9b04-1b6b5e617606/327631ed-b242-499d-854c-7f681e3bd80e/00cb560b-b02f-42f4-9ee8-09089f5ae590) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:09.253218 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.329107 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Pushing event updated for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:09.335237 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:09.527562 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] For port 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:09.527926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.572127 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.587599 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Attempting to bind port 350a3cff-ca12-4254-a4cf-07a695eb5914 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:09.588056 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Attempting to bind port 350a3cff-ca12-4254-a4cf-07a695eb5914 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:09.590807 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Attempting to bind port 350a3cff-ca12-4254-a4cf-07a695eb5914 on network a5c48f0c-499f-4abd-b04c-ed57773dbaef {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:09.608193 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 5, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:09.608694 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Checking segment: {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:09.609254 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.609749 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Bound using segment: {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:09.610419 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Bound port: 350a3cff-ca12-4254-a4cf-07a695eb5914, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '62003036-d603-4e95-8128-f4d87aace384', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 97, 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:09.611261 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.753355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] For port 350a3cff-ca12-4254-a4cf-07a695eb5914, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:09.757500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Calling delete_port for 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 owned by network:router_gateway {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:09.757688 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] For port 350a3cff-ca12-4254-a4cf-07a695eb5914, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=350a3cff-ca12-4254-a4cf-07a695eb5914,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:09.805395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Transition to ACTIVE for port object 00cb560b-b02f-42f4-9ee8-09089f5ae590 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:09.814454 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:09.819679 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:09.822386 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-74aa628a-006f-4995-a2a0-5554d92ff326 req-1918cb6a-6a24-4010-b764-18d06751df96 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.0614765 May 24 10:05:09.920358 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:09.921129 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:09.999915 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Pushing event updated for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:10.010729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.019745 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:10.069461 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e98a9e5c-edee-4d26-b4ec-fc0780b3c106 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:10.130079 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.172552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:10.219108 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:10.230735 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:10.233465 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:10.518175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:05:10.589341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.644314 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:05:10.647702 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 97fff5ef-d87b-48e1-9d52-2ff0a4597cc4 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:05:10.647702 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b154e178-27fc-4709-b3e0-94ee2928fd20 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:05:10.649510 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:10.701613 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: b154e178-27fc-4709-b3e0-94ee2928fd20 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:05:10.702203 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b154e178-27fc-4709-b3e0-94ee2928fd20 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:05:10.735288 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.666s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:10.740043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-00ae66df-e654-4075-b286-e76e7d67645c tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.570s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:10.791255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-efa472ef-b470-4d87-a9c7-af51213e8ed6 None None] Provisioning for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:10.809207 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-efa472ef-b470-4d87-a9c7-af51213e8ed6 None None] Provisioning complete for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:10.809694 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-efa472ef-b470-4d87-a9c7-af51213e8ed6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.874938 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-efa472ef-b470-4d87-a9c7-af51213e8ed6 None None] Port 00cb560b-b02f-42f4-9ee8-09089f5ae590 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:05:10.886748 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:10.887411 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.901069 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 4, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '350a3cff-ca12-4254-a4cf-07a695eb5914', 'name': '', 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'mac_address': 'fa:16:3e:5e:56:30', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_interface', 'standard_attr_id': 445, 'fixed_ips': [{'subnet_id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:03Z', 'updated_at': '2021-05-24T10:05:09Z', 'revision_number': 4, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'subnets': [{'id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:10.906071 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a7d5ae2-49bf-470c-bf51-5207b34847e6 None None] The sync data for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi is: [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:05:04Z', 'revision_number': 4, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '350a3cff-ca12-4254-a4cf-07a695eb5914', 'name': '', 'network_id': 'a5c48f0c-499f-4abd-b04c-ed57773dbaef', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'mac_address': 'fa:16:3e:5e:56:30', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_interface', 'standard_attr_id': 445, 'fixed_ips': [{'subnet_id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:03Z', 'updated_at': '2021-05-24T10:05:09Z', 'revision_number': 4, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'subnets': [{'id': 'ffc35f12-deef-45c7-9064-c4d2fba75cd3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:10.950684 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:10.986308 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Pushing event deleted for resources: {'Port': ['ID=97fff5ef-d87b-48e1-9d52-2ff0a4597cc4,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:11.004599 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.008184 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.269s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:11.139666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60a029eb-d58d-4a04-9cc0-987c181cabfa req-12ec66c0-9d8c-4d1c-b52a-9e4ebbaa7237 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:11.148667 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-60a029eb-d58d-4a04-9cc0-987c181cabfa req-12ec66c0-9d8c-4d1c-b52a-9e4ebbaa7237 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 932 time: 0.1448257 May 24 10:05:11.203177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.203775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Fanout notify agent at l3_agent the message router_deleted on router 8eac1314-02e8-4975-a4e9-f33c258a1bd0 {{(pid=109113) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:05:11.213256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.217423 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] 10.222.0.67 "DELETE /v2.0/routers/8eac1314-02e8-4975-a4e9-f33c258a1bd0 HTTP/1.1" status: 204 len: 168 time: 3.1904008 May 24 10:05:11.218211 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57592) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:11.223374 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-60a029eb-d58d-4a04-9cc0-987c181cabfa req-f732ff69-1ec1-4cd8-947e-bc034ccc48e6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:11.225419 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 2.7767162 May 24 10:05:11.240069 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-60a029eb-d58d-4a04-9cc0-987c181cabfa req-f732ff69-1ec1-4cd8-947e-bc034ccc48e6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:11.247805 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-60a029eb-d58d-4a04-9cc0-987c181cabfa req-f732ff69-1ec1-4cd8-947e-bc034ccc48e6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0978048 May 24 10:05:11.281381 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Request body: {'port': {'device_id': '812a56d8-d836-470a-b693-994765367aec', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:11.380021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Deleting subnet 98c67f17-d401-4c79-8169-5da43506ba3c {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:05:11.406704 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.490299 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.502400 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:11.630602 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.154954 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:12.163727 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.182572 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Transition to ACTIVE for port object 00cb560b-b02f-42f4-9ee8-09089f5ae590 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:12.266026 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.272801 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b074aa3c-fb52-4287-a362-9010e6af37e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:12.327276 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=98c67f17-d401-4c79-8169-5da43506ba3c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:12.329687 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.334939 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:12.339945 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-cc45dbb5-3eb4-442e-a45f-198bb4a0a0c4 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/98c67f17-d401-4c79-8169-5da43506ba3c HTTP/1.1" status: 204 len: 168 time: 1.1209357 May 24 10:05:12.344150 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57608) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:12.361478 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.367144 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-10da9b7f-5697-472c-92da-6072c2b6fb28 tempest-ServerRescueNegativeTestJSON-895971592 tempest-ServerRescueNegativeTestJSON-895971592-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:12.542187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.558420 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 350a3cff-ca12-4254-a4cf-07a695eb5914 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:12.654710 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:12.667414 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.300s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:12.748756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.764269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > segment before_delete 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.764752 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.765715 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.766175 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > port after_create 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.766603 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > port after_delete 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.767042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > port after_update 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.767470 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > segment after_delete 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.767881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.768281 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.768695 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet after_update 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.780421 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 350a3cff-ca12-4254-a4cf-07a695eb5914 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:12.787333 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 350a3cff-ca12-4254-a4cf-07a695eb5914 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:12.787723 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.849838 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.850332 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.850759 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet before_create 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.851181 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Subscribe: > subnet before_update 55550000 {{(pid=109112) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 10:05:12.851711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['b0fff367-5ea3-4833-b1bf-44e4dd284af1']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:12.877498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:12.878423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'bb46500f-61f2-431d-97f0-2ba79eaa0fd2') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:12.891885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.920190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.934230 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Releasing vlan 71 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:05:12.935571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:12.975653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Attempting to bind port 00cb560b-b02f-42f4-9ee8-09089f5ae590 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:12.976167 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Attempting to bind port 00cb560b-b02f-42f4-9ee8-09089f5ae590 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:12.976940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Attempting to bind port 00cb560b-b02f-42f4-9ee8-09089f5ae590 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: from cryptography.utils import int_from_bytes May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: from cryptography.utils import int_from_bytes May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:05:12.980939 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:05:12.988800 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Port 00cb560b-b02f-42f4-9ee8-09089f5ae590 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:05:12.989413 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Attempting to bind port 00cb560b-b02f-42f4-9ee8-09089f5ae590 on network c1fb75f5-deee-4236-9b04-1b6b5e617606 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:12.997319 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 4, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:12.998454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Checking segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:12.999045 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Bound using segment: {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:12.999665 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Bound port: 00cb560b-b02f-42f4-9ee8-09089f5ae590, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f1abdab4-bf27-4888-b131-574a25f02d59', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 90, 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:13.000196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.033126 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ab02970-b758-4081-8a40-d535310cc339 req-d6eb355c-4bcd-40b6-aa00-20f71d119f05 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:13.034275 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6ab02970-b758-4081-8a40-d535310cc339 req-d6eb355c-4bcd-40b6-aa00-20f71d119f05 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Synchronizing usage tracker for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:13.038646 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.122888 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 350a3cff-ca12-4254-a4cf-07a695eb5914, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=350a3cff-ca12-4254-a4cf-07a695eb5914,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:13.135983 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:13.140363 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:13.184891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.189265 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:13.254582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.259823 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:13.305484 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:13.308960 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:13.321891 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.328235 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cb74dd18-f277-4c19-a7ca-8023f8559e21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:13.337379 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6ab02970-b758-4081-8a40-d535310cc339 req-d6eb355c-4bcd-40b6-aa00-20f71d119f05 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Unset dirty status for tenant:c42fc1bb2ae6477fa7b7c9261f2d50bb on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:13.346799 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6ab02970-b758-4081-8a40-d535310cc339 req-d6eb355c-4bcd-40b6-aa00-20f71d119f05 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.8902848 May 24 10:05:13.359729 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.402645 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:13.403464 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:13.404908 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] 10.222.0.95 "PUT /v2.0/ports/00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 1094 time: 2.1314893 May 24 10:05:13.406934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:13.409795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Pushing event deleted for resources: {'Network': ['ID=b0fff367-5ea3-4833-b1bf-44e4dd284af1,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:13.417145 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:13.448757 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] 10.222.0.67 "DELETE /v2.0/networks/b0fff367-5ea3-4833-b1bf-44e4dd284af1 HTTP/1.1" status: 204 len: 168 time: 1.1042399 May 24 10:05:13.476020 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6ab02970-b758-4081-8a40-d535310cc339 req-bf93da3a-2763-4b6c-9f5f-37da2d3ca843 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1279919 May 24 10:05:13.481342 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:13.488418 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.229s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:13.590333 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'status': 'completed', 'code': 200} May 24 10:05:13.650454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3f45d381-c170-4bfa-a026-cb88d31fce4e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:13.652115 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-3f45d381-c170-4bfa-a026-cb88d31fce4e service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1097 time: 0.1220038 May 24 10:05:13.742961 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-8a2a09f6-a93d-41cc-950c-d7b27420ae6b service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 217 time: 0.0719759 May 24 10:05:13.849617 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-23a418ed-8a19-4115-ad70-2749511ba456 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:13.851263 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-23a418ed-8a19-4115-ad70-2749511ba456 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0963371 May 24 10:05:13.936408 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57626) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:13.954150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-475de9f3-f2f1-4d74-b153-13ddffd1d286 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:13.955644 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-475de9f3-f2f1-4d74-b153-13ddffd1d286 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0971258 May 24 10:05:14.010942 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6c023584-d44e-403e-b859-bf01fb383c7c admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b08409afad374227ace1cdf31e4e9412&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0728989 May 24 10:05:14.013374 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57630) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:14.078472 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:14.106300 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:14.107622 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-056048a8-1cb5-490f-a819-e254a92bcab4 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1395380 May 24 10:05:14.218564 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:14.223498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-59068be1-36e6-454a-b19e-a22931d20548 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:14.237008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fe64f46b-8749-477e-bbac-58ca92a691c2,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:14.243724 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:14.268662 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6faf4435-73f9-4b1f-b80f-a04753de30ba req-debda228-63c4-4147-b9e4-f7970307690d service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1535339 May 24 10:05:14.338287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:14.339944 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/fe64f46b-8749-477e-bbac-58ca92a691c2 HTTP/1.1" status: 204 len: 168 time: 0.3252327 May 24 10:05:14.497659 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 350a3cff-ca12-4254-a4cf-07a695eb5914 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:14.566810 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66fa423e-700e-436f-b214-448ab3e94930 req-8e6fb2ed-16aa-46e5-8f9d-9662c9ea1d73 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:14.569667 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-8e6fb2ed-16aa-46e5-8f9d-9662c9ea1d73 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1097 time: 0.1588564 May 24 10:05:14.620000 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 350a3cff-ca12-4254-a4cf-07a695eb5914 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:14.620621 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:14.838312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66fa423e-700e-436f-b214-448ab3e94930 req-0460a16f-8ea2-4917-97ea-fab056a41670 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:14.839779 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-0460a16f-8ea2-4917-97ea-fab056a41670 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.2563133 May 24 10:05:14.968621 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1cb9c53-933a-4cef-8166-5f1375b7e458 req-a4e33565-9315-4c6c-8049-483f213f2800 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:14.970267 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b1cb9c53-933a-4cef-8166-5f1375b7e458 req-a4e33565-9315-4c6c-8049-483f213f2800 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.3378274 May 24 10:05:14.983704 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-e5036eb3-1981-443b-b0e2-dc0d32ac9851 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 217 time: 0.1361518 May 24 10:05:15.134501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66fa423e-700e-436f-b214-448ab3e94930 req-80d3b3cb-5cbd-4816-bc2b-abc2154bb8f8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:15.136744 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-80d3b3cb-5cbd-4816-bc2b-abc2154bb8f8 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.1008301 May 24 10:05:15.151006 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b1cb9c53-933a-4cef-8166-5f1375b7e458 req-244494f4-bfc7-464e-a13e-e0d62c557638 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1657565 May 24 10:05:15.251043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66fa423e-700e-436f-b214-448ab3e94930 req-d31fba59-99be-4e62-ba76-84aec9b1df4c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:15.252621 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-d31fba59-99be-4e62-ba76-84aec9b1df4c service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1004694 May 24 10:05:15.561271 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-df1f7bd8-dcdf-41f7-ba78-c44c74339fe0 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2999473 May 24 10:05:15.753369 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66fa423e-700e-436f-b214-448ab3e94930 req-6bf4e49e-5ae0-4a79-844a-421b776efb1f service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1837757 May 24 10:05:15.768643 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8c28d30-314b-49e7-ab6b-6c3037845652 req-86ac0ab8-d82e-45eb-85b8-489d6c530e50 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:15.770426 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e8c28d30-314b-49e7-ab6b-6c3037845652 req-86ac0ab8-d82e-45eb-85b8-489d6c530e50 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 945 time: 0.1995699 May 24 10:05:15.820744 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e8c28d30-314b-49e7-ab6b-6c3037845652 req-04184a7f-f9bb-41f0-8a7c-dcd2e4b60aab tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448968 May 24 10:05:16.411775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef360321-2b91-4773-aad1-d7b56290abb6 req-505dae1d-7c89-4cee-bb27-610af4b4d1a6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:16.413939 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ef360321-2b91-4773-aad1-d7b56290abb6 req-505dae1d-7c89-4cee-bb27-610af4b4d1a6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1062148 May 24 10:05:16.487864 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ef360321-2b91-4773-aad1-d7b56290abb6 req-35c4bd85-8d90-40ed-b9d0-c7b5b2b82e44 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0653889 May 24 10:05:17.023160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad02bd41-9a03-44af-ab42-5a2dcd2d0aae req-2d6c3752-98f2-4d3f-a62f-0338f9255b6d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:17.025458 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ad02bd41-9a03-44af-ab42-5a2dcd2d0aae req-2d6c3752-98f2-4d3f-a62f-0338f9255b6d tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 945 time: 0.0965128 May 24 10:05:17.076163 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ad02bd41-9a03-44af-ab42-5a2dcd2d0aae req-0ad38d68-851d-4c54-a9d4-fd27bc38d6dd tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/security-groups?id=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424545 May 24 10:05:17.162613 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:05:17.306675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-274edfd3-54e6-4358-8212-6d7c900dacfe tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:17.309569 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-274edfd3-54e6-4358-8212-6d7c900dacfe tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 945 time: 0.1490295 May 24 10:05:17.349556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:17.364196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:05:17.428737 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:17.462341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:17.508578 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:17.579529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Deleting port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:05:17.579529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:17.614240 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:17.617536 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:17.746095 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b136788a-14a5-4b9c-830e-2679a0bff34b req-40b1dd6c-0bbc-4f2f-9a95-5b0ad057b681 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:17.749351 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b136788a-14a5-4b9c-830e-2679a0bff34b req-40b1dd6c-0bbc-4f2f-9a95-5b0ad057b681 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 945 time: 0.1290798 May 24 10:05:17.759955 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:17.762422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] For port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,segment=NetworkSegment(3ddab603-0e7a-4025-8390-7039b3ef3fd1),segment_id=3ddab603-0e7a-4025-8390-7039b3ef3fd1)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:17.769741 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:17.770557 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:17.813253 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b136788a-14a5-4b9c-830e-2679a0bff34b req-11f70184-d8e7-44b7-aa41-af01d410b74e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0560403 May 24 10:05:17.844876 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['00cb560b-b02f-42f4-9ee8-09089f5ae590'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:05:17.911347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Calling delete_port for 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:18.115256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:18.121395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ee0d5c14-adab-48b3-a1b2-248cfcc199ca admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:18.170759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Pushing event deleted for resources: {'Port': ['ID=086b9a1c-9348-4b08-b6f7-f934b5e8aaa4,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:18.182232 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.182618 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.182997 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.183355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.183708 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.184064 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.184456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.184852 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.185225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.185575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.185921 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.186263 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:18.190728 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:18.301963 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ff76e39a-c7c3-48af-93e9-1222edc70d54 req-63b5b4a4-a0e6-46b7-ab81-facb2e870dda tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.0836315 May 24 10:05:18.307625 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] No DVR routers for this DVR port 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:05:18.310292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:18.313188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'name': 'network-vif-deleted', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:18.343694 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['086b9a1c-9348-4b08-b6f7-f934b5e8aaa4'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:05:18.365666 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '8315e706-66b1-49e6-a581-061abaf6c6db', 'name': 'network-vif-deleted', 'tag': '086b9a1c-9348-4b08-b6f7-f934b5e8aaa4', 'status': 'completed', 'code': 200} May 24 10:05:18.373805 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7f194c0f-2b4b-4d3a-bff7-ddf77ecc2ffb req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.95 "DELETE /v2.0/ports/086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 HTTP/1.1" status: 204 len: 173 time: 1.0520074 May 24 10:05:19.034008 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-58b48746-7882-4136-a199-58f67a700f3f req-aa331c5e-c61e-44c2-8569-8ec55f84945d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:19.035649 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-58b48746-7882-4136-a199-58f67a700f3f req-aa331c5e-c61e-44c2-8569-8ec55f84945d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 945 time: 0.0905256 May 24 10:05:19.107236 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-58b48746-7882-4136-a199-58f67a700f3f req-14e9b7a9-b725-4806-afdc-fd07b453275c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0609753 May 24 10:05:19.196378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:19.480598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3a97d8b4-6b09-4be6-bb74-ca09f09f88ed req-c37846e7-a5c0-4670-8e0f-7414bc1b2683 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Synchronizing usage tracker for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:19.595769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:19.601016 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:19.601464 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:19.724584 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:19.752783 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:19.762639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:19.790420 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3a97d8b4-6b09-4be6-bb74-ca09f09f88ed req-c37846e7-a5c0-4670-8e0f-7414bc1b2683 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Unset dirty status for tenant:0fa87500244642ee805dba47191bc7e6 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:19.796882 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3a97d8b4-6b09-4be6-bb74-ca09f09f88ed req-c37846e7-a5c0-4670-8e0f-7414bc1b2683 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] 10.222.0.67 "GET /v2.0/ports?device_id=8315e706-66b1-49e6-a581-061abaf6c6db HTTP/1.1" status: 200 len: 210 time: 0.3615093 May 24 10:05:19.849685 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'code': 200} May 24 10:05:19.933299 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:19.934166 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:05:19.958086 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:20.015687 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:05:20.016055 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 086b9a1c-9348-4b08-b6f7-f934b5e8aaa4 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:05:20.035991 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:20.038905 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:20.156657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:20.318820 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03ce05dc-e863-4405-955d-1d3d6514885c req-b892b709-4ffc-4cf8-832f-2396001a5108 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:20.320482 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-03ce05dc-e863-4405-955d-1d3d6514885c req-b892b709-4ffc-4cf8-832f-2396001a5108 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1099 time: 0.0925975 May 24 10:05:20.330041 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc399b07-e731-4f77-a1fd-77e5ad9b1f5f req-86019bf3-b2b7-4411-be68-996aee3a9f3c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:20.331660 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dc399b07-e731-4f77-a1fd-77e5ad9b1f5f req-86019bf3-b2b7-4411-be68-996aee3a9f3c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.0971098 May 24 10:05:20.374060 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-03ce05dc-e863-4405-955d-1d3d6514885c req-43dbb3d1-a6dc-43af-bd13-96818b7c4fc0 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0460725 May 24 10:05:20.393241 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dc399b07-e731-4f77-a1fd-77e5ad9b1f5f req-4258ab74-7db3-418d-b49a-6d28effd272b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0546541 May 24 10:05:21.612566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5ebc8e1-a85f-4305-b856-aa6c77a09fb3 req-a83db3a1-d321-4228-a896-329a26fff290 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:21.614398 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a5ebc8e1-a85f-4305-b856-aa6c77a09fb3 req-a83db3a1-d321-4228-a896-329a26fff290 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1074827 May 24 10:05:21.688504 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a5ebc8e1-a85f-4305-b856-aa6c77a09fb3 req-76202149-9b28-478b-a96a-9d9c12f86529 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0619879 May 24 10:05:21.705190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-1014e851-b330-4987-bbd7-cca8b3b80682 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:21.706814 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-1014e851-b330-4987-bbd7-cca8b3b80682 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 815 time: 0.2629638 May 24 10:05:21.722450 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-b73c79d5-f7f9-4464-affe-c41dabfb0ba7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/quotas/4b47c7cc67c545eab92ea86cde1abac7 HTTP/1.1" status: 200 len: 375 time: 0.0100281 May 24 10:05:21.812046 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-7f6af3d7-93bf-48eb-8f09-2b7b54c705d3 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0840814 May 24 10:05:21.952974 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-88e583a7-f73d-43ea-951c-d5ec54282d31 service neutron] 10.222.0.67 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1287572 May 24 10:05:22.088046 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-f23d052d-f05d-4eb5-b225-9e82155dd440 service neutron] 10.222.0.67 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1293318 May 24 10:05:22.337734 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a0c83432-98ee-4486-acfb-7513d2056818 req-51f19814-13e7-4558-a25a-1a98c226856c tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:22.411866 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-a0c83432-98ee-4486-acfb-7513d2056818 req-51f19814-13e7-4558-a25a-1a98c226856c tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:22.413036 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a0c83432-98ee-4486-acfb-7513d2056818 req-51f19814-13e7-4558-a25a-1a98c226856c tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.1202402 May 24 10:05:22.903542 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f44096a8-3f28-42a3-b32f-f7fe0d528dd2 req-ea8dd1e5-1bdf-4f00-9b8a-90c41242f835 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:22.905005 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f44096a8-3f28-42a3-b32f-f7fe0d528dd2 req-ea8dd1e5-1bdf-4f00-9b8a-90c41242f835 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.0905364 May 24 10:05:22.951676 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f44096a8-3f28-42a3-b32f-f7fe0d528dd2 req-42d43c3d-a1ad-4da4-81d5-87adce44904d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407498 May 24 10:05:23.576415 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ecbec640-6f86-4947-8163-8382dd0183bd req-87a09814-7799-4443-b091-4bcd964441c1 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.0465310 May 24 10:05:24.163782 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57bffa86-a0d3-4c34-93f8-1786d16c2a05 req-08cf3d64-c0c7-43e4-be8e-38807cff06fc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:24.165801 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-57bffa86-a0d3-4c34-93f8-1786d16c2a05 req-08cf3d64-c0c7-43e4-be8e-38807cff06fc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1018670 May 24 10:05:24.222044 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-57bffa86-a0d3-4c34-93f8-1786d16c2a05 req-745c4458-90fa-4848-9d73-287ab676396f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0472510 May 24 10:05:24.325210 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6cb0c537-aa18-4d3d-8811-2b6a9bd66308 req-97afc8c2-e83b-4577-a41b-266aeac6ed18 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:24.327353 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6cb0c537-aa18-4d3d-8811-2b6a9bd66308 req-97afc8c2-e83b-4577-a41b-266aeac6ed18 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0978358 May 24 10:05:24.378705 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6cb0c537-aa18-4d3d-8811-2b6a9bd66308 req-c0d7c75e-4312-4cce-8d9c-553d774f59e1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0446448 May 24 10:05:24.727244 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-570cfc72-51bc-496f-aa53-b2777b55be76 req-14fd8f5b-4994-42f9-8d66-3b040da65062 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.0434775 May 24 10:05:25.432468 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31c812ec-7942-4f82-a0c9-84023e5540d2 req-17ff5016-4828-4f28-be51-3429d9387f76 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:25.433955 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-31c812ec-7942-4f82-a0c9-84023e5540d2 req-17ff5016-4828-4f28-be51-3429d9387f76 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.0852742 May 24 10:05:25.479196 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-31c812ec-7942-4f82-a0c9-84023e5540d2 req-b1792685-d795-4a57-a3cf-a214e8bba0aa tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391917 May 24 10:05:25.574842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c26a89aa-28e3-4161-94ab-39650674255d req-0080395d-1b8b-4cb4-830b-adcd3ab58c13 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:25.576519 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c26a89aa-28e3-4161-94ab-39650674255d req-0080395d-1b8b-4cb4-830b-adcd3ab58c13 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.0843964 May 24 10:05:25.621785 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c26a89aa-28e3-4161-94ab-39650674255d req-fbb029ab-330c-402c-9344-6aaf462a93c5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0393836 May 24 10:05:25.665241 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49825b1d-cc19-4fd2-9995-cda9569f3902 req-f22bccf7-7e41-40d5-9b5f-a4c475baef6f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:25.667906 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-49825b1d-cc19-4fd2-9995-cda9569f3902 req-f22bccf7-7e41-40d5-9b5f-a4c475baef6f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.0811961 May 24 10:05:25.676446 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57824) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:25.681696 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Request body: {'floatingip': {'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:25.764076 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:25.898124 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-49fe7dde-d450-4953-9dcf-300b05fc7daf req-32ce7bde-9630-41b2-96be-74cebc2293e9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.0481255 May 24 10:05:26.549200 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:26.611628 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:26.832695 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Floating IP 9453c2e9-84b1-4b24-856e-dc0745e22ee7 associated. External IP: 172.27.0.60, port: 00cb560b-b02f-42f4-9ee8-09089f5ae590. May 24 10:05:27.001139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19503741-5575-4168-b867-92a7258ccb08 req-5b130042-a476-4c0a-a08d-29e634f51723 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:27.002727 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-19503741-5575-4168-b867-92a7258ccb08 req-5b130042-a476-4c0a-a08d-29e634f51723 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2511976 May 24 10:05:27.047460 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f83fecaa-758e-4dcc-80cc-710ae044544e req-24d0c6f0-2a78-4a8e-a0cc-1dafe0ee6de0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.0438468 May 24 10:05:27.139958 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-19503741-5575-4168-b867-92a7258ccb08 req-2c17213f-e693-4181-997c-704c61aafb62 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1254022 May 24 10:05:27.262687 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:05:27.445862 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:05:27.450831 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:27.455316 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:05:27.465483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:27.466226 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:27.506881 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-77cbca28-7939-494d-bb7a-f2041da311e8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "PUT /v2.0/floatingips/9453c2e9-84b1-4b24-856e-dc0745e22ee7 HTTP/1.1" status: 200 len: 991 time: 1.8293290 May 24 10:05:27.507603 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:27.563007 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'status': 'completed', 'code': 200} May 24 10:05:27.647661 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:27.853611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153a827d-c715-4a22-bd42-c288bea234da req-99ac7218-cc8e-49e6-a2bb-67dcaee19183 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:27.855106 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-99ac7218-cc8e-49e6-a2bb-67dcaee19183 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1099 time: 0.0830073 May 24 10:05:27.939139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-26331876-3a05-46fa-8807-7b66ab5e767a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:27.940724 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-26331876-3a05-46fa-8807-7b66ab5e767a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 815 time: 0.1807394 May 24 10:05:27.960871 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'port': {'device_id': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'admin_state_up': True, 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:28.023572 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153a827d-c715-4a22-bd42-c288bea234da req-3c4bb393-1866-4b5c-bb13-84d2f5e1902e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:28.025638 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-3c4bb393-1866-4b5c-bb13-84d2f5e1902e service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.1639817 May 24 10:05:28.091682 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9453c2e9-84b1-4b24-856e-dc0745e22ee7', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.60', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 439, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '812a56d8-d836-470a-b693-994765367aec', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:04:57Z', 'updated_at': '2021-05-24T10:05:26Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:28.097160 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:05:28.108717 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:28.109756 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:28.162818 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-c5197756-43f1-4567-8da9-9e678aecd720 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 999 time: 0.1304615 May 24 10:05:28.318409 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:28.318705 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:28.340465 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:28.353521 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153a827d-c715-4a22-bd42-c288bea234da req-11f15bd3-3b56-4bbf-81ad-0d5e34bd9ca3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:28.356275 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-11f15bd3-3b56-4bbf-81ad-0d5e34bd9ca3 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.1762791 May 24 10:05:28.484132 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153a827d-c715-4a22-bd42-c288bea234da req-ef0df832-d3d9-42de-a29c-f86a204651fb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:28.485674 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-ef0df832-d3d9-42de-a29c-f86a204651fb service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1197104 May 24 10:05:28.511009 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Made reservation on behalf of 4b47c7cc67c545eab92ea86cde1abac7 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:05:28.518625 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:28.667978 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c7c139a5-7041-4f2d-b9f6-6d454068cacb req-072344fb-af55-40d6-a54e-539d5bd8a2c5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.3482335 May 24 10:05:28.742313 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-0a6dce01-ea43-41e8-a51e-3f061d631f0b service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2479184 May 24 10:05:28.938944 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9453c2e9-84b1-4b24-856e-dc0745e22ee7', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.60', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 439, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '812a56d8-d836-470a-b693-994765367aec', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:04:57Z', 'updated_at': '2021-05-24T10:05:26Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:28.944983 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f3a37a1e-fa1a-45d1-9677-6eed79298828 None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9453c2e9-84b1-4b24-856e-dc0745e22ee7', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'floating_ip_address': '172.27.0.60', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 439, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '812a56d8-d836-470a-b693-994765367aec', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:04:57Z', 'updated_at': '2021-05-24T10:05:26Z', 'revision_number': 1, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:29.215181 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-a3489e7a-aae1-4c34-ab1b-f3da18a9bdc2 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4599087 May 24 10:05:29.260859 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-7cb580f4-b923-4b82-b459-2b1aa667354e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:29.274803 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-7cb580f4-b923-4b82-b459-2b1aa667354e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 0.2253964 May 24 10:05:29.406531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Allocated IP 172.20.0.7 (d34a1523-ea50-45ad-a897-459ffff7af90/a1ecfc88-eaf2-426e-bb90-f6a474fb5991/fb9be3d2-4022-41e3-825a-ddf6079de0ba) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:29.427869 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d3f91712-be8c-4b19-adb7-06b535f594aa None None] New status for floating IP 9453c2e9-84b1-4b24-856e-dc0745e22ee7: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:05:29.451293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d0cc0d8-6447-42ac-9e54-e5bb2aa2e2cb req-cecb30fa-2168-43b2-83d3-c64d3f80b5bc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:29.464121 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2d0cc0d8-6447-42ac-9e54-e5bb2aa2e2cb req-cecb30fa-2168-43b2-83d3-c64d3f80b5bc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 945 time: 1.1037993 May 24 10:05:29.621807 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Deleting port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:05:29.622172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:29.631675 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2d0cc0d8-6447-42ac-9e54-e5bb2aa2e2cb req-66a2d17a-ed36-402e-8eff-b7a368bb55f4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=c864d029-e730-4251-9cfd-084868a9275d&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1665459 May 24 10:05:29.817907 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] For port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:29.818505 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:29.880374 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:29.923069 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Transition to ACTIVE for port object fb9be3d2-4022-41e3-825a-ddf6079de0ba will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:30.057846 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a4c64df2-ad63-4477-be3a-685e56f6031c req-3e58fb50-3084-4433-a3c1-1c9cfbe064b7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.1110504 May 24 10:05:30.125392 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.157629 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9b2fb525-d57d-4210-a387-8a68dbcb7b30 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:30.266620 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Network d34a1523-ea50-45ad-a897-459ffff7af90 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:30.270438 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:30.273084 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:30.344743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:30.348384 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.191s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:30.374551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:30.375494 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.494241 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.5422246 May 24 10:05:30.522989 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Request body: {'port': {'device_id': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:30.610472 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.611340 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.611340 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Floating IP c963c3b2-33f2-42e6-89bf-8d9a8954d2af disassociated. External IP: 172.27.0.64, port: b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f. May 24 10:05:30.611621 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Calling delete_port for b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f owned by compute:nova {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:30.683765 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 57916) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:30.864947 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-952eb04d-943c-4c6e-8d82-ba57faf5e97a None None] Provisioning for port fb9be3d2-4022-41e3-825a-ddf6079de0ba completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:30.873093 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-952eb04d-943c-4c6e-8d82-ba57faf5e97a None None] Provisioning complete for port fb9be3d2-4022-41e3-825a-ddf6079de0ba triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:30.878060 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-952eb04d-943c-4c6e-8d82-ba57faf5e97a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.949289 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.984051 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5206c27f-ff09-4e22-b8f6-1955be4084f8 req-6feeea99-2fc1-45ea-9955-1cf7403ddc93 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.1054635 May 24 10:05:30.986002 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:30.995775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-59e5e666-8ebd-45ae-a79d-13a590e0c03c tempest-VolumesActionsTest-1211716999 tempest-VolumesActionsTest-1211716999-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:31.061527 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'Port': ['ID=b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:31.076837 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:31.078040 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-952eb04d-943c-4c6e-8d82-ba57faf5e97a None None] Port fb9be3d2-4022-41e3-825a-ddf6079de0ba cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:05:31.166270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.166950 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.167473 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.167980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.168590 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.169205 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.169722 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.170234 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.170753 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.171257 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.171767 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.172269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:31.183336 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.188s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:31.205320 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:31.214923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Usage tracker for resource:security_group_rule and tenant:28f8f4692ea4414fac3731cfc6bc454c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:31.225441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:31.352530 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:31.384094 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:05:31.839325 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:05:31.853549 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:05:31.875912 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8acf2025-b134-425f-a51e-f1056b627e5a req-15395b3f-b915-4221-8d2d-8d28e7e452af tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:31.877115 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-8acf2025-b134-425f-a51e-f1056b627e5a req-15395b3f-b915-4221-8d2d-8d28e7e452af tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:31.889642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:31.896248 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:32.053960 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] No DVR routers for this DVR port b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f on host hv-cinder-79123 {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:05:32.058548 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:32.063546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'name': 'network-vif-deleted', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:32.108443 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:32.112316 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-8acf2025-b134-425f-a51e-f1056b627e5a req-15395b3f-b915-4221-8d2d-8d28e7e452af tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:32.113513 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8acf2025-b134-425f-a51e-f1056b627e5a req-15395b3f-b915-4221-8d2d-8d28e7e452af tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 931 time: 0.8671045 May 24 10:05:32.138654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id fb9be3d2-4022-41e3-825a-ddf6079de0ba {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:32.173280 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c8dc5be5-bfb2-4a0a-99c2-3ea93be65cee req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "DELETE /v2.0/ports/b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f HTTP/1.1" status: 204 len: 173 time: 2.8860810 May 24 10:05:32.202272 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:32.207883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:32.234666 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e3b7abc5-61f7-4170-bbac-f6d91c4ae06d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:32.236521 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:32.423838 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8acf2025-b134-425f-a51e-f1056b627e5a req-ea1a10fa-5728-43fb-bbe1-454a5c8dc23a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2805207 May 24 10:05:32.439253 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Transition to ACTIVE for port object fb9be3d2-4022-41e3-825a-ddf6079de0ba will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:32.463621 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=28f8f4692ea4414fac3731cfc6bc454c&name=default HTTP/1.1" status: 200 len: 2979 time: 1.7710538 May 24 10:05:32.465241 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57938) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:32.470142 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2bc61a42-7319-40f3-aaa9-37bf2df9bfe5,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:32.473975 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.242s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:32.537519 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:32.549972 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-75a5920b-9c88-4b76-a87e-40eaab7d93a0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:32.826084 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:32.831842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.287s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:33.006410 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-80fdbe55-fbe1-4789-89d6-123d4da6b7b6 req-029fdb63-cb4b-4975-b0b2-ad71cb2be27e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.1178076 May 24 10:05:33.083571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.140459 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:33.154347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Attempting to bind port fb9be3d2-4022-41e3-825a-ddf6079de0ba on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:33.154642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Attempting to bind port fb9be3d2-4022-41e3-825a-ddf6079de0ba by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:33.156149 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Attempting to bind port fb9be3d2-4022-41e3-825a-ddf6079de0ba on network d34a1523-ea50-45ad-a897-459ffff7af90 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:33.170767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Port fb9be3d2-4022-41e3-825a-ddf6079de0ba on network d34a1523-ea50-45ad-a897-459ffff7af90 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:05:33.173019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Attempting to bind port fb9be3d2-4022-41e3-825a-ddf6079de0ba on network d34a1523-ea50-45ad-a897-459ffff7af90 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:33.179252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.193105 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b5b44b45-4a2f-48d5-851a-222d48691c7e', 'name': 'network-vif-deleted', 'tag': 'b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f', 'status': 'completed', 'code': 200} May 24 10:05:33.197416 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 5, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:33.197865 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Checking segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:33.199160 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:05:33.200103 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Bound using segment: {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:33.200769 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Bound port: fb9be3d2-4022-41e3-825a-ddf6079de0ba, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:33.203882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.255311 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:05:33.339449 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.348247 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d933f414-e565-4a8f-8dcc-ac1d85080e70 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:33.383838 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2bc61a42-7319-40f3-aaa9-37bf2df9bfe5,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:33.388829 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:33.399925 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.402164 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/2bc61a42-7319-40f3-aaa9-37bf2df9bfe5 HTTP/1.1" status: 204 len: 168 time: 0.9343007 May 24 10:05:33.415766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:33.436048 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:33.510581 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:05:33.511481 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:33.540213 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.547995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:33.680650 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:05:33.681123 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b53384d6-69bf-43ea-b7ca-1bf1a5e0a26f not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:05:33.696813 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9255285d-3ee9-426a-a984-589d092e272f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.904875 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:33.905724 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:33.909488 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] 10.222.0.95 "PUT /v2.0/ports/fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 1093 time: 3.3890655 May 24 10:05:33.909784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5410fe21-f0a4-4e0b-96f0-1c3b52622cef req-c7794f31-198f-4c75-a865-c499b3bf13f4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:33.909784 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5410fe21-f0a4-4e0b-96f0-1c3b52622cef req-c7794f31-198f-4c75-a865-c499b3bf13f4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.2055337 May 24 10:05:33.921512 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:33.972198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:33.988390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.440s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:34.010841 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:05:34.074262 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-5410fe21-f0a4-4e0b-96f0-1c3b52622cef req-c6fa0d1e-71df-4437-a23c-f30335787e2d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:34.163760 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:05:34.213780 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-5410fe21-f0a4-4e0b-96f0-1c3b52622cef req-c6fa0d1e-71df-4437-a23c-f30335787e2d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:34.223987 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5410fe21-f0a4-4e0b-96f0-1c3b52622cef req-c6fa0d1e-71df-4437-a23c-f30335787e2d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2868662 May 24 10:05:34.303808 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-fceb73c4-b63e-4d11-af1a-4afc41fbfb79 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:34.310439 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-fceb73c4-b63e-4d11-af1a-4afc41fbfb79 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1096 time: 0.2832654 May 24 10:05:34.553237 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-343f42a2-c0d2-4b1d-b149-359cec396e3f service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 217 time: 0.2207010 May 24 10:05:34.561762 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-24a2bc77-f2ef-49f4-81db-a6d027369478 req-e428cb31-d890-4142-ab48-f19afee2c9fb tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=b5b44b45-4a2f-48d5-851a-222d48691c7e HTTP/1.1" status: 200 len: 210 time: 0.1052773 May 24 10:05:34.694119 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-d100013d-bbf8-48f5-a94d-fa21bd009d31 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:34.697517 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-d100013d-bbf8-48f5-a94d-fa21bd009d31 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.1347880 May 24 10:05:34.972568 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:34.977891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4068a777-5427-4464-84f6-94f20ff5b930 None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:05:35.350090 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a886da1d-08ea-4288-ba17-4ab04d5ad92e None None] New status for floating IP c963c3b2-33f2-42e6-89bf-8d9a8954d2af: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:05:35.717844 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4111d67f-5c69-4762-9bb2-2d32f86c6b02 req-dd002f87-2c52-497f-a979-05d2f27140c3 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:35.725902 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4111d67f-5c69-4762-9bb2-2d32f86c6b02 req-dd002f87-2c52-497f-a979-05d2f27140c3 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.2505567 May 24 10:05:35.810279 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4111d67f-5c69-4762-9bb2-2d32f86c6b02 req-31175c56-96b2-4e35-a083-16bc41021255 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0825605 May 24 10:05:35.885849 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-444afaec-400a-4dd0-bf25-fb45313a17a7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:35.927860 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:36.109026 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 57998) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:36.185253 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:36.216095 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:36.385061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:36.393163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f8bcb1ab-e029-4262-962b-ff9c8da17861 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:36.410002 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=c864d029-e730-4251-9cfd-084868a9275d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:36.412580 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:36.486803 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:36.487912 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "DELETE /v2.0/security-groups/c864d029-e730-4251-9cfd-084868a9275d HTTP/1.1" status: 204 len: 168 time: 0.3778250 May 24 10:05:36.490670 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58002) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:36.507481 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.api.v2.resource [None req-16a314ec-8119-40c6-a8b1-9aa475f1224e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] show failed (client error): The resource could not be found. May 24 10:05:36.511484 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-16a314ec-8119-40c6-a8b1-9aa475f1224e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups/c864d029-e730-4251-9cfd-084868a9275d HTTP/1.1" status: 404 len: 347 time: 0.0169425 May 24 10:05:36.516065 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58006) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:36.803460 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Deleting port f981d8ee-64c4-43a8-ac03-61f54f5cc98d {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:05:36.804630 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.078292 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.078628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.217376 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Calling delete_port for f981d8ee-64c4-43a8-ac03-61f54f5cc98d owned by network:floatingip {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:05:37.218893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.269561 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f699dd-6e87-49fa-be59-2717925eb8b5 req-bb89b511-34f2-4b4e-bd1d-940f4d801a8d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:37.271155 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b5f699dd-6e87-49fa-be59-2717925eb8b5 req-bb89b511-34f2-4b4e-bd1d-940f4d801a8d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.2579916 May 24 10:05:37.405793 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b5f699dd-6e87-49fa-be59-2717925eb8b5 req-a51876ab-c723-46fd-a514-28f9f900aca7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1181304 May 24 10:05:37.475770 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.480714 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:37.527557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'Port': ['ID=f981d8ee-64c4-43a8-ac03-61f54f5cc98d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:37.555079 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:37.623002 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.637059 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:37.638259 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "DELETE /v2.0/floatingips/c963c3b2-33f2-42e6-89bf-8d9a8954d2af HTTP/1.1" status: 204 len: 168 time: 1.1256263 May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58030) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group': {'name': 'tempest-securitygroup--245601581', 'description': 'tempest-description--1182146368'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:38.448507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:38.628823 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:38.712410 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:38.723676 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:38.728270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:38.737793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:38.795067 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72bdaa93-6392-49c2-be5f-f268f3f29053 req-ee43bc9f-13a7-487c-b3c0-7c218947e49f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:38.800345 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72bdaa93-6392-49c2-be5f-f268f3f29053 req-ee43bc9f-13a7-487c-b3c0-7c218947e49f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.1382623 May 24 10:05:38.872155 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:38.875587 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72bdaa93-6392-49c2-be5f-f268f3f29053 req-42f07389-c1c1-4d42-bf17-7072320c7e27 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0714455 May 24 10:05:38.978727 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:39.653668 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:39.663398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0f1dc555-9cc1-45a5-9794-ba897ba1cd97 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:39.746822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroup': ['ID=fcf796c6-f87f-462a-9699-d361158092a8,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:39.750308 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:39.807309 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:39.809021 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 1.4320328 May 24 10:05:39.812856 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58044) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:39.818287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group_rule': {'security_group_id': 'fcf796c6-f87f-462a-9699-d361158092a8', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:39.824827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:39.829672 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:39.832745 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:39.839663 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:39.943772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.088557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.170031 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-025cc3fc-34a2-4907-9964-39a1d1bf3ac4 req-8809a43d-6566-4704-a700-d546df0299d5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:40.171833 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-025cc3fc-34a2-4907-9964-39a1d1bf3ac4 req-8809a43d-6566-4704-a700-d546df0299d5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.1236186 May 24 10:05:40.173210 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.175248 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:40.293883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=bf3cd612-ef6e-4557-a573-d5901c108372,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:40.305546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:40.331023 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-025cc3fc-34a2-4907-9964-39a1d1bf3ac4 req-9a54186c-336c-483c-91be-e2d3c19d3d70 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1460040 May 24 10:05:40.401146 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.402344 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.5884678 May 24 10:05:40.406903 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58056) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:40.412249 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'security_group_rule': {'security_group_id': 'fcf796c6-f87f-462a-9699-d361158092a8', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:40.418722 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:40.423223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:security_group_rule and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:40.426652 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:40.433669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:40.486359 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:05:40.546766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.619558 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.670628 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.672739 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f60ae259-cb7c-4dbd-bc18-a53d1b48b1c9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:40.696766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=df5865ce-579b-41c7-b116-3ef88743733e,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:40.702223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:40.755618 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:40.756839 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.3489625 May 24 10:05:40.759890 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58062) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:40.764990 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:40.771049 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:40.774983 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Usage tracker for resource:floatingip and tenant:3f91ad6930924a8d81ee21ec1c490828 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:40.778010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:40.784781 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:40.864120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.003962 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.148199 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Allocated IP 172.27.0.96 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/d23a3389-d1d4-4243-951e-265997c76053) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:41.302813 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.401613 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.406260 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:41.468173 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:41.471307 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:41.475502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:41.516275 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=d23a3389-d1d4-4243-951e-265997c76053,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:41.520400 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:41.528088 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c65952a7-a03c-4959-b7c1-eea569cabc1b req-27ba53d5-784d-4bc5-992a-0217b199cac4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:41.530632 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c65952a7-a03c-4959-b7c1-eea569cabc1b req-27ba53d5-784d-4bc5-992a-0217b199cac4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.0871053 May 24 10:05:41.567864 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.576837 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c65952a7-a03c-4959-b7c1-eea569cabc1b req-543face1-6e29-49c5-840d-37e17e76b520 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398161 May 24 10:05:41.809778 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:41.880372 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-419ccc20-b751-499c-8be1-acdec8b2f797 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:41.881870 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-419ccc20-b751-499c-8be1-acdec8b2f797 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.3032925 May 24 10:05:42.116250 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-fe815729-9554-4e42-af4b-4eb3dbab6c9c service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2193229 May 24 10:05:42.159903 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:42.244397 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:42.250602 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e4c0f8d6-3f03-468f-bd3a-7da253118684 tempest-VolumesSnapshotTestJSON-1598006192 tempest-VolumesSnapshotTestJSON-1598006192-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:42.487363 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-416ded4c-5d45-4b68-977e-12c4af661ab8 req-bfc9dae8-b59c-4147-b7e6-1f888458eaa1 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3556278 May 24 10:05:42.504596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=d23a3389-d1d4-4243-951e-265997c76053,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:42.504952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.254s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:42.519449 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:42.667863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:42.669514 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.9087200 May 24 10:05:42.778253 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-c4dd64b2-b599-499f-8e9b-e0fdf6052329 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:42.779804 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-c4dd64b2-b599-499f-8e9b-e0fdf6052329 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1096 time: 0.1462948 May 24 10:05:42.846011 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-337d0856-2d18-42cb-b17e-1e1f6b340f2b req-4803f71c-0770-4913-9bb0-f342b2a0d4f5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:42.847499 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-337d0856-2d18-42cb-b17e-1e1f6b340f2b req-4803f71c-0770-4913-9bb0-f342b2a0d4f5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.1488070 May 24 10:05:42.985447 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-337d0856-2d18-42cb-b17e-1e1f6b340f2b req-03f6d69d-9806-41d7-a72f-67d9546e1736 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0795615 May 24 10:05:42.992979 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-1bc8bf66-55b4-4595-85d5-64b686342551 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:42.994496 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-1bc8bf66-55b4-4595-85d5-64b686342551 service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.2027102 May 24 10:05:43.072361 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-db9a0793-4746-4765-9b9d-b37390c4697b service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 217 time: 0.0708883 May 24 10:05:43.142885 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-5180f9a8-90fd-4149-a6b7-9aed49fa29ea service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:43.144362 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-5180f9a8-90fd-4149-a6b7-9aed49fa29ea service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.0634587 May 24 10:05:43.158834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-dd00db3d-1609-4251-996c-f143f446ed5e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:43.232292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-3c56e8ce-9338-45e3-9678-a2cd857a4a0a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:43.233782 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-3c56e8ce-9338-45e3-9678-a2cd857a4a0a service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0824392 May 24 10:05:43.258254 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-dd00db3d-1609-4251-996c-f143f446ed5e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:43.259498 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-dd00db3d-1609-4251-996c-f143f446ed5e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-securitygroup--245601581&fields=id&tenant_id=3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 267 time: 0.1436179 May 24 10:05:43.377783 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-91c3054a-79f6-481c-b8a7-0c72bc39fcb7 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1364357 May 24 10:05:43.433470 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-7bbeb615-70bd-47e0-8510-66717bff05ea tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:43.435475 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-7bbeb615-70bd-47e0-8510-66717bff05ea tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 810 time: 0.1674645 May 24 10:05:43.450876 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-7ec830df-5e4a-4df1-a8d2-14ab77789ac1 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/quotas/3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 375 time: 0.0092733 May 24 10:05:43.517749 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cfc4575d-13ca-4793-8c43-ea1f9809ce8b req-39a1c36a-8249-4faf-9e27-f7fa0fe92096 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1325347 May 24 10:05:43.537328 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-82190dec-6685-418b-906d-bf8f3ca4f55c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:43.618211 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-82190dec-6685-418b-906d-bf8f3ca4f55c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:43.619753 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-82190dec-6685-418b-906d-bf8f3ca4f55c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1635890 May 24 10:05:43.812227 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-7004e092-561a-45e3-8593-be8847363fd6 service neutron] 10.222.0.67 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1802068 May 24 10:05:43.944314 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-8bf7a10b-c754-4f15-a787-bd0c5e55cf88 service neutron] 10.222.0.67 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1261930 May 24 10:05:44.232928 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-20b3d754-90c5-4ee3-8b0f-7a200713ac6a req-a21db0e4-0b1b-4be2-a794-995ef7268462 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 210 time: 0.0438311 May 24 10:05:44.274193 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3ec0d2a-44e3-4110-86c7-acee449228bc req-6f448e80-591d-4891-8cc1-a4be71b0dd21 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:44.275748 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b3ec0d2a-44e3-4110-86c7-acee449228bc req-6f448e80-591d-4891-8cc1-a4be71b0dd21 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.0925643 May 24 10:05:44.330186 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b3ec0d2a-44e3-4110-86c7-acee449228bc req-49e45b65-b8a5-4986-a698-18934366785e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0483615 May 24 10:05:45.283113 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device fb9be3d2-4022-41e3-825a-ddf6079de0ba details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:05:45.484091 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:45.499278 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'port_id': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'mac_address': 'fa:16:3e:e9:3d:dc', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:05:45.544027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d437034-077d-4df8-a6f3-138251b7078d req-400fc218-0297-4ca0-870f-3dd8589988f2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:45.546158 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5d437034-077d-4df8-a6f3-138251b7078d req-400fc218-0297-4ca0-870f-3dd8589988f2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 943 time: 0.0934458 May 24 10:05:45.549110 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1611b70a-ee70-4a6b-be49-2f9f1dd797b4 req-b07209c7-64af-4099-b384-2f9ab3f91169 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 210 time: 0.0507817 May 24 10:05:45.571450 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:45.595121 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id fb9be3d2-4022-41e3-825a-ddf6079de0ba {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:45.616407 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:45.618358 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5d437034-077d-4df8-a6f3-138251b7078d req-617f7b89-d2f8-455f-8255-3fe3da36bb98 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663018 May 24 10:05:45.744229 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:45.746009 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:45.815821 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:45.823306 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.823664 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.824006 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.824342 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.824675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.825042 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.825908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.826237 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.826563 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.826887 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.827210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.827529 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:05:45.831869 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:45.979184 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['fb9be3d2-4022-41e3-825a-ddf6079de0ba'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:05:46.724628 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-27716674-d9af-488b-9fa7-76d8e57d6c10 req-88102480-b3c7-4166-9e0d-6af4a2f71017 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 210 time: 0.0566199 May 24 10:05:46.860601 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dfb718ec-8013-4c9b-88ba-11377fa9061a req-6aa42aa7-fc3f-4205-95b5-fbac8cdc030b tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:46.863329 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dfb718ec-8013-4c9b-88ba-11377fa9061a req-6aa42aa7-fc3f-4205-95b5-fbac8cdc030b tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 944 time: 0.0920799 May 24 10:05:46.869097 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-f722c183-5b1d-4b18-a7db-ae5985d6a2e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:46.870616 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-f722c183-5b1d-4b18-a7db-ae5985d6a2e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 810 time: 0.2296610 May 24 10:05:46.913706 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dfb718ec-8013-4c9b-88ba-11377fa9061a req-d7b95479-3b84-47a4-a6f1-f2ed892bb3e0 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0435421 May 24 10:05:46.925171 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-7e6b75e1-594e-4cb6-b944-45d1ba83212d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=3f91ad6930924a8d81ee21ec1c490828 HTTP/1.1" status: 200 len: 379 time: 0.0448768 May 24 10:05:46.934466 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'port': {'device_id': '91e8cab0-6add-4148-873b-9f1564abc530', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'admin_state_up': True, 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'security_groups': ['fcf796c6-f87f-462a-9699-d361158092a8']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:47.082880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:47.095090 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:47.184215 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Made reservation on behalf of 3f91ad6930924a8d81ee21ec1c490828 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:05:47.188302 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.299244 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device fb9be3d2-4022-41e3-825a-ddf6079de0ba up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:47.375615 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Allocated IP 172.20.0.6 (e28486c2-462f-406b-8c64-3246ae76f56c/59e96feb-f5d8-4b1e-b08d-b93c34f5461c/08d1ba31-f09e-4772-a2c3-16a65ba1761c) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:47.401255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port fb9be3d2-4022-41e3-825a-ddf6079de0ba completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:47.406952 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port fb9be3d2-4022-41e3-825a-ddf6079de0ba triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:47.407409 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.540891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.570495 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:47.588228 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:47.615066 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'code': 200} May 24 10:05:47.663970 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.666221 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:47.680270 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.697182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Transition to ACTIVE for port object 08d1ba31-f09e-4772-a2c3-16a65ba1761c will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:47.733770 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:47.736967 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:47.779665 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:47.786826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0cebfd78-c45b-48c0-b983-fcf3eaa11c45 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:47.861783 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Network e28486c2-462f-406b-8c64-3246ae76f56c is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:05:47.868256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:47.871918 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:05:47.974802 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event updated for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:47.990473 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.204s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:48.036199 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:48.037231 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:48.051325 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-486f8ff4-7414-440b-8dca-67744338ad9a req-e8b076e0-7e1c-4012-bdf2-ae9d2ef321ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:48.053144 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-486f8ff4-7414-440b-8dca-67744338ad9a req-e8b076e0-7e1c-4012-bdf2-ae9d2ef321ba tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 931 time: 0.1873930 May 24 10:05:48.084802 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f2cc84f-0f3b-4f05-8575-3617345e0988 req-bdd9b1fe-f139-4b32-9fa1-a5f7e721c46a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:48.087312 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f2cc84f-0f3b-4f05-8575-3617345e0988 req-bdd9b1fe-f139-4b32-9fa1-a5f7e721c46a service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1098 time: 0.1565442 May 24 10:05:48.165662 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-486f8ff4-7414-440b-8dca-67744338ad9a req-c08d1974-d3bf-4652-877f-3a051b4df47d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0951307 May 24 10:05:48.185885 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.2555127 May 24 10:05:48.206000 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Request body: {'port': {'device_id': '91e8cab0-6add-4148-873b-9f1564abc530', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:48.210501 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f2cc84f-0f3b-4f05-8575-3617345e0988 req-dd7f5225-fab0-4368-80e4-64353ed78a8d service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1149619 May 24 10:05:48.237849 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26df479d-acfd-40cf-9173-721c4f518278 req-b875ede7-3d0c-4aa4-9ec5-159f5e91b709 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:48.239657 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-26df479d-acfd-40cf-9173-721c4f518278 req-b875ede7-3d0c-4aa4-9ec5-159f5e91b709 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1187353 May 24 10:05:48.299636 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-26df479d-acfd-40cf-9173-721c4f518278 req-13f414e9-5144-4c97-af91-82b879328f7a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0540373 May 24 10:05:48.310335 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-d47b9073-74e6-46c5-9340-0e2bc51a2c1a None None] Provisioning for port 08d1ba31-f09e-4772-a2c3-16a65ba1761c completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:48.316816 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-d47b9073-74e6-46c5-9340-0e2bc51a2c1a None None] Provisioning complete for port 08d1ba31-f09e-4772-a2c3-16a65ba1761c triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:48.317232 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d47b9073-74e6-46c5-9340-0e2bc51a2c1a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:48.334046 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:48.421356 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d47b9073-74e6-46c5-9340-0e2bc51a2c1a None None] Port 08d1ba31-f09e-4772-a2c3-16a65ba1761c cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:05:48.700376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 08d1ba31-f09e-4772-a2c3-16a65ba1761c {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:48.707735 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:48.722568 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Transition to ACTIVE for port object 08d1ba31-f09e-4772-a2c3-16a65ba1761c will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:48.863874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:48.867750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:49.027024 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Pushing event updated for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:49.042988 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:49.116189 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Attempting to bind port 08d1ba31-f09e-4772-a2c3-16a65ba1761c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:05:49.116525 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Attempting to bind port 08d1ba31-f09e-4772-a2c3-16a65ba1761c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:05:49.116973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Attempting to bind port 08d1ba31-f09e-4772-a2c3-16a65ba1761c on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:49.122767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Port 08d1ba31-f09e-4772-a2c3-16a65ba1761c on network e28486c2-462f-406b-8c64-3246ae76f56c not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:05:49.122767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Attempting to bind port 08d1ba31-f09e-4772-a2c3-16a65ba1761c on network e28486c2-462f-406b-8c64-3246ae76f56c {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:05:49.128440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 5, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:05:49.128772 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Checking segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:49.129182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Bound using segment: {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:05:49.129616 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Bound port: 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '25064880-e56c-4a2f-b56c-21997837afc6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 61, 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:05:49.129967 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:49.230074 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:05:49.233585 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:05:49.303695 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:49.310152 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-358e983a-a89b-45c7-93e2-a4c0b12b1fbe service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:49.363720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3badcb10-66f8-44d3-9c51-a6de324b4f68 req-c7f051e6-cc89-469d-939d-89c4cd1ad376 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:49.365603 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3badcb10-66f8-44d3-9c51-a6de324b4f68 req-c7f051e6-cc89-469d-939d-89c4cd1ad376 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 943 time: 0.0931065 May 24 10:05:49.381350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:49.381350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:49.381350 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] 10.222.0.95 "PUT /v2.0/ports/08d1ba31-f09e-4772-a2c3-16a65ba1761c HTTP/1.1" status: 200 len: 1093 time: 1.1857517 May 24 10:05:49.382330 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:49.420930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Pushing event updated for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:49.424304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:49.444069 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3badcb10-66f8-44d3-9c51-a6de324b4f68 req-7d8ef568-6d01-450b-b45d-e5fc436671df tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0623543 May 24 10:05:49.506020 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'status': 'completed', 'code': 200} May 24 10:05:49.600591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79a08e84-4212-4ff0-984e-da645319f82a req-027361b4-eb08-462f-902b-f062734fb2c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:49.602229 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-79a08e84-4212-4ff0-984e-da645319f82a req-027361b4-eb08-462f-902b-f062734fb2c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1600978 May 24 10:05:49.636586 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-f3c175ec-b01b-46ab-92c5-d5403fa1eeba service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:49.638136 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-f3c175ec-b01b-46ab-92c5-d5403fa1eeba service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 1096 time: 0.1357996 May 24 10:05:49.736633 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-a110ef14-1471-488e-ab21-a8fa5c2f3eb7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c HTTP/1.1" status: 200 len: 217 time: 0.0853772 May 24 10:05:49.741716 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-79a08e84-4212-4ff0-984e-da645319f82a req-d4e61e78-d798-4ded-bd31-7f145e3a0ca1 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1312747 May 24 10:05:49.813076 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-50a22abb-1a40-4cf9-a51a-989bc2af4f41 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:49.814688 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-50a22abb-1a40-4cf9-a51a-989bc2af4f41 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.0700121 May 24 10:05:50.056698 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-54409bd3-18b4-4381-aa9f-bbfdf7422cd2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.058249 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-54409bd3-18b4-4381-aa9f-bbfdf7422cd2 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2362921 May 24 10:05:50.070796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c2b8570-76d1-4747-85d8-87719dc6649c req-662e9e4c-4d1c-40e9-bf63-8578f8440d2d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.072302 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c2b8570-76d1-4747-85d8-87719dc6649c req-662e9e4c-4d1c-40e9-bf63-8578f8440d2d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.2161252 May 24 10:05:50.129038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58264) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:50.132287 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.208258 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7c2b8570-76d1-4747-85d8-87719dc6649c req-760387d1-d8ff-4da2-b710-59a56dec9e85 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1163657 May 24 10:05:50.243499 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-e1a1b427-303e-42d9-99a0-0423b10798b8 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1702070 May 24 10:05:50.253976 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.268789 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:50.272921 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Usage tracker for resource:security_group_rule and tenant:837d6473cffb46cfa3bc5fdfa890b4f7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:50.276141 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:50.282790 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:50.380344 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.391923 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c60e781b-9604-416f-b026-c3e4ce0f2e6c req-4fda4090-be74-475b-bf9c-db32ed5625dd service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1414030 May 24 10:05:50.423606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.431232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-237b89e6-6fc4-48a3-9a4c-f9b69ee1aa0a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:50.472631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=db847b95-37fe-4a31-bfdb-0d2478f9d1d9,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:50.479804 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=837d6473cffb46cfa3bc5fdfa890b4f7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3494630 May 24 10:05:50.480987 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:50.481483 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58276) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:50.557081 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.584396 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.663454 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8046b024-575c-4976-b976-8c595ebe475c req-69fdcf6a-22ef-43b2-bf29-eea97a39b26d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.664975 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-69fdcf6a-22ef-43b2-bf29-eea97a39b26d service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 1096 time: 0.1395185 May 24 10:05:50.680195 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-852fa639-f167-4af8-8c06-9ec052c1b109 req-497177bb-e0ca-4116-83fa-432530d56e42 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.682225 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-852fa639-f167-4af8-8c06-9ec052c1b109 req-497177bb-e0ca-4116-83fa-432530d56e42 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1207271 May 24 10:05:50.707430 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff14c4fa-1994-4437-8d27-92e04e034b14 req-910b0ddf-75fa-49bc-8645-dbffaf497e1e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.709791 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ff14c4fa-1994-4437-8d27-92e04e034b14 req-910b0ddf-75fa-49bc-8645-dbffaf497e1e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 943 time: 0.1175673 May 24 10:05:50.768243 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.774988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-96cd6ed3-1ee2-402f-a01d-17d6c4c06f48 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:50.795279 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ff14c4fa-1994-4437-8d27-92e04e034b14 req-f43e784f-1115-4f31-a351-d9f528a1cd47 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0714312 May 24 10:05:50.798731 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=db847b95-37fe-4a31-bfdb-0d2478f9d1d9,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:50.807391 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:50.808060 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/db847b95-37fe-4a31-bfdb-0d2478f9d1d9 HTTP/1.1" status: 204 len: 168 time: 0.3252528 May 24 10:05:50.810413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3aef7a0a-d4d4-44cc-b6f6-1c0efd1324a5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:50.847010 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-852fa639-f167-4af8-8c06-9ec052c1b109 req-e4e33af8-c138-46e4-b4cd-376a6a2ead00 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1441429 May 24 10:05:50.901696 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8046b024-575c-4976-b976-8c595ebe475c req-48e71a79-ae05-4f7e-820f-ae77bdc33694 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:50.903710 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-48e71a79-ae05-4f7e-820f-ae77bdc33694 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.2197504 May 24 10:05:50.980438 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-4c6d45fb-da25-424b-a730-125d0de5aca9 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c HTTP/1.1" status: 200 len: 217 time: 0.0684438 May 24 10:05:51.056427 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8046b024-575c-4976-b976-8c595ebe475c req-0ebe7f0d-6d3e-4256-8986-b11ddc024e7c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:51.057980 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-0ebe7f0d-6d3e-4256-8986-b11ddc024e7c service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.0704367 May 24 10:05:51.167873 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8046b024-575c-4976-b976-8c595ebe475c req-a52757b2-8420-4552-b9c2-cdd13029dafe service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:51.170014 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-a52757b2-8420-4552-b9c2-cdd13029dafe service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1026218 May 24 10:05:51.310396 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-12d0640c-a010-4d0e-8eef-bbb713c532e2 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1326530 May 24 10:05:51.454541 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8046b024-575c-4976-b976-8c595ebe475c req-fc2b7a43-96ec-436e-97b6-c8db517dcf2b service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1348152 May 24 10:05:51.989190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e879549-0827-41a1-97c7-7a1ecd687734 req-4f46b544-23ee-457f-88fe-377da92ff652 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:51.990440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6e879549-0827-41a1-97c7-7a1ecd687734 req-4f46b544-23ee-457f-88fe-377da92ff652 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:05:52.063764 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-6e879549-0827-41a1-97c7-7a1ecd687734 req-4f46b544-23ee-457f-88fe-377da92ff652 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:05:52.065038 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6e879549-0827-41a1-97c7-7a1ecd687734 req-4f46b544-23ee-457f-88fe-377da92ff652 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 943 time: 0.1611214 May 24 10:05:52.117953 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6e879549-0827-41a1-97c7-7a1ecd687734 req-3f742513-3cf9-4129-8436-bdf88ca8f486 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0470755 May 24 10:05:53.321589 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6e6ef88-c839-4289-8a93-deb5a199135f req-ea77a9a3-9a4b-4424-b002-545d1ac600ad tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:53.323112 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e6e6ef88-c839-4289-8a93-deb5a199135f req-ea77a9a3-9a4b-4424-b002-545d1ac600ad tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 943 time: 0.0845726 May 24 10:05:53.337008 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 08d1ba31-f09e-4772-a2c3-16a65ba1761c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:05:53.370541 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e6e6ef88-c839-4289-8a93-deb5a199135f req-da569bfe-d1b0-48cc-ba34-5a6ddeaee01e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412617 May 24 10:05:53.480803 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:53.494478 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'port_id': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'mac_address': 'fa:16:3e:12:b6:9a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 61, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:05:53.554500 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:53.571402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 08d1ba31-f09e-4772-a2c3-16a65ba1761c {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:05:53.588339 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:53.686481 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:53.688229 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:53.770109 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:53.775021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:53.921818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['08d1ba31-f09e-4772-a2c3-16a65ba1761c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:05:53.949210 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58418) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:54.013904 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Deleting subnet 9ee9ac65-ecfe-47d4-bbac-4561dabf43e6 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:05:54.017180 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.072876 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.129413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.544697 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.555620 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:54.596221 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=9ee9ac65-ecfe-47d4-bbac-4561dabf43e6,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:54.599886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.603489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:54.611327 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/9ee9ac65-ecfe-47d4-bbac-4561dabf43e6 HTTP/1.1" status: 204 len: 168 time: 0.6609790 May 24 10:05:54.614071 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58430) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:54.667969 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03f49583-8a93-463d-8da4-1c0f940b5078 req-613cafe7-4324-439f-ba50-fbc52df6afb6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:54.669399 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-03f49583-8a93-463d-8da4-1c0f940b5078 req-613cafe7-4324-439f-ba50-fbc52df6afb6 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 944 time: 0.1748130 May 24 10:05:54.755794 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.781780 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-03f49583-8a93-463d-8da4-1c0f940b5078 req-3aa8666d-2fe8-40fa-a88c-6ace7c9e88f0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0650382 May 24 10:05:54.901990 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.920777 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['2852c3dc-6508-45a7-9576-a36787475d18']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:54.936373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '3ddab603-0e7a-4025-8390-7039b3ef3fd1') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:54.936678 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '3ddab603-0e7a-4025-8390-7039b3ef3fd1') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:05:54.952730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.969681 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.975835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Releasing vlan 63 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:05:54.976218 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:05:54.990127 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:05:55.115133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.118936 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b2443bd5-dd65-4612-873e-0c25f1332d88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:55.134864 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.140776 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Pushing event deleted for resources: {'Network': ['ID=2852c3dc-6508-45a7-9576-a36787475d18,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:55.144847 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:55.224764 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5f7ef546-98fc-45d3-a783-783a1ec6e84e admin admin] 10.222.0.67 "DELETE /v2.0/networks/2852c3dc-6508-45a7-9576-a36787475d18 HTTP/1.1" status: 204 len: 168 time: 0.6096613 May 24 10:05:55.347893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 08d1ba31-f09e-4772-a2c3-16a65ba1761c up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:05:55.509187 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 08d1ba31-f09e-4772-a2c3-16a65ba1761c completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:05:55.514544 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 08d1ba31-f09e-4772-a2c3-16a65ba1761c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:05:55.514931 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.634323 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58442) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:55.646072 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.675360 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:05:55.689992 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:05:55.702796 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-51aa5ea4-f2b4-4e7d-a68f-3dc60d343c6c admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=0fa87500244642ee805dba47191bc7e6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0671153 May 24 10:05:55.705540 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58448) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:55.728693 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'code': 200} May 24 10:05:55.772209 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.784121 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.788300 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:55.799493 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.862675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:55.865917 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:55.938654 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:55.943606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:55.957515 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:55.959995 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:55.990823 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5af2773a-617e-4a5a-9a4a-ea16d9dc8393 req-d4f849c8-9573-41b8-8f1a-04db4f6e4dee tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:55.992510 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5af2773a-617e-4a5a-9a4a-ea16d9dc8393 req-d4f849c8-9573-41b8-8f1a-04db4f6e4dee tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.0904715 May 24 10:05:56.041667 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5af2773a-617e-4a5a-9a4a-ea16d9dc8393 req-cbf13a30-c130-41c5-9c05-fb3e844a26eb tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415666 May 24 10:05:56.059428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:56.060596 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/0c64e03c-8d04-4bdd-881a-c6ff3ecadd1c HTTP/1.1" status: 204 len: 168 time: 0.3541751 May 24 10:05:56.148569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66bf3b75-6c0e-48fa-9028-29d56bd75816 req-10d60737-ac4f-4d55-8e6f-a5ff47aa6233 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:56.150091 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66bf3b75-6c0e-48fa-9028-29d56bd75816 req-10d60737-ac4f-4d55-8e6f-a5ff47aa6233 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 1098 time: 0.0707827 May 24 10:05:56.193781 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-66bf3b75-6c0e-48fa-9028-29d56bd75816 req-308f41aa-3f62-4562-9edb-fc09b0f8fe47 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0363319 May 24 10:05:57.244532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58482) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:57.248721 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c92484ae-6858-4ccd-aa11-9099a6b1cb8c req-e6ff1159-518a-425b-bc5a-8085035a318e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:57.250270 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c92484ae-6858-4ccd-aa11-9099a6b1cb8c req-e6ff1159-518a-425b-bc5a-8085035a318e tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.0931969 May 24 10:05:57.298514 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c92484ae-6858-4ccd-aa11-9099a6b1cb8c req-f0dc43df-b23e-44db-9dfb-9e882a525745 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419533 May 24 10:05:57.366001 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1087220685-network', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:57.371030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:57.374933 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Usage tracker for resource:network and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:57.377865 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:57.383787 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:57.526525 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Made reservation on behalf of ce96e2adbf10489aae7640c84f398b29 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:05:57.529513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.542990 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.558064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:57.562765 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:57.565891 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:57.572945 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:57.662666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.737667 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.740717 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b0a88547-f3e0-4833-b28e-2468f00bab3d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:57.775194 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 81} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:05:57.783969 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=11b49c10-dd6e-4ac3-84b3-9623e9146b6d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:57.787566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:57.790480 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.790835 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Added segment c69a7001-1e8e-470a-bfb0-2489247a3dd6 of type vlan for network 9a3ab75c-0613-43fc-b406-9209bc3e8724 May 24 10:05:57.818806 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.914977 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.916566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:57.917184 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:57.922669 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:58.023883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Pushing event updated for resources: {'Network': ['ID=9a3ab75c-0613-43fc-b406-9209bc3e8724,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:58.026564 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:58.033874 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:05:58.053278 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.8074765 May 24 10:05:58.055982 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58494) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:58.062272 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Request body: {'subnet': {'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1087220685-subnet', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:58.067134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:58.071238 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Usage tracker for resource:subnet and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:58.074232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:58.080315 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:58.171458 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Made reservation on behalf of ce96e2adbf10489aae7640c84f398b29 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:05:58.173785 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:58.501456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a2352db-b141-42b2-afca-606c43fc3329 req-5f215c93-4ca0-4162-bb83-e92eb723f271 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:58.503048 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a2352db-b141-42b2-afca-606c43fc3329 req-5f215c93-4ca0-4162-bb83-e92eb723f271 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.0888457 May 24 10:05:58.556475 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a2352db-b141-42b2-afca-606c43fc3329 req-04147bd0-0e1f-4650-82a5-1a1702a5b750 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0473068 May 24 10:05:58.666085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:58.672313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c2385ae8-f3f7-4563-a643-4280fab410fc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:58.867079 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Pushing event updated for resources: {'Subnet': ['ID=0f050b21-5ec6-4623-a5c0-4e203399a27b,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:05:58.872528 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:05:58.890769 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Checking segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:05:58.891131 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Network hostable DHCP agents. Network: 9a3ab75c-0613-43fc-b406-9209bc3e8724, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:05:58.920430 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d494b4b-c358-48e5-b61f-414824d913e4 req-85f6735a-142c-476b-bfc1-7b4f89dd6b88 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:58.921989 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7d494b4b-c358-48e5-b61f-414824d913e4 req-85f6735a-142c-476b-bfc1-7b4f89dd6b88 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.1497269 May 24 10:05:58.930401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58508) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:58.936235 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Request body: {'floatingip': {'port_id': '08d1ba31-f09e-4772-a2c3-16a65ba1761c'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:58.992870 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:05:59.019273 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.139142 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 9a3ab75c-0613-43fc-b406-9209bc3e8724) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:05:59.147999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-12aac402-6716-4e06-a69b-7bad62841d17 None None] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:05:59.154039 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:05:59.154640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.295885 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-2c158af3-6e5d-4cb3-bced-c728f8054eed admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.2388139 May 24 10:05:59.299382 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58512) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:05:59.306029 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:05:59.312693 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:05:59.319396 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Usage tracker for resource:router and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:05:59.330367 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:05:59.337762 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:05:59.379207 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-9a3ab75c-0613-43fc-b406-9209bc3e8724', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:05:59.380296 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.412614 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.413694 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.432006 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.557092 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Allocated IP 172.20.0.2 (9a3ab75c-0613-43fc-b406-9209bc3e8724/0f050b21-5ec6-4623-a5c0-4e203399a27b/7ade3842-cd45-419d-9045-8626ba1fdbe0) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:05:59.844037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.864692 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Transition to ACTIVE for port object 7ade3842-cd45-419d-9045-8626ba1fdbe0 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:59.885690 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Transition to ACTIVE for port object 7ade3842-cd45-419d-9045-8626ba1fdbe0 will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:05:59.907400 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.937923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.950670 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:05:59.957642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:05:59.972986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.030330 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:00.033832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:00.041246 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:00.043960 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Pushing event updated for resources: {'Port': ['ID=7ade3842-cd45-419d-9045-8626ba1fdbe0,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:00.048727 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:00.145267 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Attempting to bind port 7ade3842-cd45-419d-9045-8626ba1fdbe0 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:00.145659 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Attempting to bind port 7ade3842-cd45-419d-9045-8626ba1fdbe0 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:00.146008 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Attempting to bind port 7ade3842-cd45-419d-9045-8626ba1fdbe0 on network 9a3ab75c-0613-43fc-b406-9209bc3e8724 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:00.151623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 5, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:00.152314 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Checking segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:00.152436 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.152814 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Bound using segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:00.153509 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Bound port: 7ade3842-cd45-419d-9045-8626ba1fdbe0, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:00.153897 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.274108 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] For port 7ade3842-cd45-419d-9045-8626ba1fdbe0, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:00.277472 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] For port 7ade3842-cd45-419d-9045-8626ba1fdbe0, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=7ade3842-cd45-419d-9045-8626ba1fdbe0,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:00.285139 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Floating IP 9480b242-ba4a-420b-bc39-da7c46eed2f8 associated. External IP: 172.27.0.96, port: 08d1ba31-f09e-4772-a2c3-16a65ba1761c. May 24 10:06:00.349419 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.354123 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:00.382860 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.421208 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.428888 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Pushing event updated for resources: {'Port': ['ID=7ade3842-cd45-419d-9045-8626ba1fdbe0,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:00.435035 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:00.606154 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Allocated IP 172.27.0.234 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/34bf5469-5bdd-4b70-b23f-5015d71654e9) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:00.710867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:06:00.841404 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.930492 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:00.936276 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.941156 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:00.956425 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:00.957201 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:00.978370 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:00.983425 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:00.998434 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d859b816-3f18-42b7-af34-e28677724523 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "PUT /v2.0/floatingips/9480b242-ba4a-420b-bc39-da7c46eed2f8 HTTP/1.1" status: 200 len: 990 time: 2.0665383 May 24 10:06:00.999287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:06:01.071337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Pushing event updated for resources: {'Port': ['ID=34bf5469-5bdd-4b70-b23f-5015d71654e9,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:01.076094 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:01.085949 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:01.087206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:01.091649 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:01.204523 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'status': 'completed', 'code': 200} May 24 10:06:01.217708 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:01.476150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-2f55cd84-0e2e-4781-b774-f133dcb7a3d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:01.484104 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-2f55cd84-0e2e-4781-b774-f133dcb7a3d8 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=3f91ad6930924a8d81ee21ec1c490828&device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 1098 time: 0.2351747 May 24 10:06:01.622112 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c27bf857-4ee6-4ac2-8890-05234eaf280f None None] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:06:01.836207 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9480b242-ba4a-420b-bc39-da7c46eed2f8', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.96', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 459, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:12:b6:9a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91e8cab0-6add-4148-873b-9f1564abc530', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:05:41Z', 'updated_at': '2021-05-24T10:05:59Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:01.850301 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:01.857982 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-02a98251-1b93-4de1-9877-4504ccafbfa0 None None] Provisioning for port 7ade3842-cd45-419d-9045-8626ba1fdbe0 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:01.874610 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:01.876213 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:01.904623 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-4edb4a64-e287-49c6-8b64-9ed2d84da7e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:01.906134 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-4edb4a64-e287-49c6-8b64-9ed2d84da7e7 service neutron] 10.222.0.95 "GET /v2.0/networks?id=e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 200 len: 910 time: 0.4177353 May 24 10:06:02.031831 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:02.119790 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-33e666d7-c62a-44dc-95b6-807fe610583f service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c HTTP/1.1" status: 200 len: 998 time: 0.1985717 May 24 10:06:02.177268 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:02.292911 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-06e3ef35-be50-41ab-ab37-0d2659ff02eb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:02.295459 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-06e3ef35-be50-41ab-ab37-0d2659ff02eb service neutron] 10.222.0.95 "GET /v2.0/subnets?id=59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 200 len: 849 time: 0.1683054 May 24 10:06:02.615525 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 7ade3842-cd45-419d-9045-8626ba1fdbe0 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:02.633637 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:02.642944 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-187cf858-4f26-4749-bc67-2babb4f57b33 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:02.644439 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-187cf858-4f26-4749-bc67-2babb4f57b33 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=e28486c2-462f-406b-8c64-3246ae76f56c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.3365157 May 24 10:06:02.660386 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2191a631-aadd-46e7-b5f0-90eb787f225c req-04182e37-a262-4735-8c0e-130edd63f82a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:02.662838 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2191a631-aadd-46e7-b5f0-90eb787f225c req-04182e37-a262-4735-8c0e-130edd63f82a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.0997760 May 24 10:06:02.720156 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2191a631-aadd-46e7-b5f0-90eb787f225c req-335ab0cc-2949-4a77-9824-7c495caf7a80 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0500069 May 24 10:06:02.759557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58548) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:02.765940 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9480b242-ba4a-420b-bc39-da7c46eed2f8', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.96', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 459, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:12:b6:9a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91e8cab0-6add-4148-873b-9f1564abc530', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:05:41Z', 'updated_at': '2021-05-24T10:05:59Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:02.773254 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd063e1-982b-4e31-aea2-599dc3a1b9aa None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '9480b242-ba4a-420b-bc39-da7c46eed2f8', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'floating_ip_address': '172.27.0.96', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'port_id': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 459, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'mac_address': 'fa:16:3e:12:b6:9a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91e8cab0-6add-4148-873b-9f1564abc530', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:05:41Z', 'updated_at': '2021-05-24T10:05:59Z', 'revision_number': 1, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:02.823394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 7ade3842-cd45-419d-9045-8626ba1fdbe0 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:02.830025 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 7ade3842-cd45-419d-9045-8626ba1fdbe0 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:02.830391 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:02.950409 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-fcbe5eb3-e357-438f-859c-7d55f96b4ee2 service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2549593 May 24 10:06:02.961762 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:02.997729 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 7ade3842-cd45-419d-9045-8626ba1fdbe0, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=7ade3842-cd45-419d-9045-8626ba1fdbe0,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:03.058686 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-32c8f248-aa86-4007-85e1-fcde24d38444 tempest-TestMinimumBasicScenario-163880250 tempest-TestMinimumBasicScenario-163880250-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:03.060630 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-32c8f248-aa86-4007-85e1-fcde24d38444 tempest-TestMinimumBasicScenario-163880250 tempest-TestMinimumBasicScenario-163880250-project-admin] Synchronizing usage tracker for tenant:f7673d41f8314855baa9c60fd5e2e1ba on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:06:03.069332 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.075769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:03.105607 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b636be5d-556e-4b1c-afc4-9ec6ef555cc2 None None] New status for floating IP 9480b242-ba4a-420b-bc39-da7c46eed2f8: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:06:03.131508 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Router 91d55ed4-1190-40a0-83e1-ece622fa9c2e is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109113) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:06:03.137193 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6de1f8c2-df0f-4a0e-b7dc-09d767fc8d15 req-c103e0e6-a5dc-4818-b382-bf2b3bed59fd service neutron] 10.222.0.95 "GET /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1764572 May 24 10:06:03.166559 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=7ade3842-cd45-419d-9045-8626ba1fdbe0,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:03.174267 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:03.178127 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-32c8f248-aa86-4007-85e1-fcde24d38444 tempest-TestMinimumBasicScenario-163880250 tempest-TestMinimumBasicScenario-163880250-project-admin] Unset dirty status for tenant:f7673d41f8314855baa9c60fd5e2e1ba on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:06:03.183217 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-32c8f248-aa86-4007-85e1-fcde24d38444 tempest-TestMinimumBasicScenario-163880250 tempest-TestMinimumBasicScenario-163880250-project-admin] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1093 time: 0.4215043 May 24 10:06:03.183993 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58556) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:03.190701 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'port_id': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'fixed_ip_address': '172.20.0.7'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:03.197591 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:03.214423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:floatingip and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:03.220232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:03.228810 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:03.280980 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8d4ce49c-d081-41e9-9f2a-c0538a18591e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.309878 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for floatingip, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.466332 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:03.486038 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Sync routers for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:03.497268 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.501191 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:03.502466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:03.586240 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.591623 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 4.2908607 May 24 10:06:03.592513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58564) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:03.683496 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Allocated IP 172.27.0.64 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/f0a1e941-ba29-43f9-8b64-db23166bf9ed) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:03.722786 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:03.860624 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.943958 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:03.949448 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:04.024046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:04.027265 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:04.032178 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:04.047260 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:01Z', 'revision_number': 3, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:00Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:04.051887 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Checking router: 91d55ed4-1190-40a0-83e1-ece622fa9c2e for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:04.052341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Updating router 91d55ed4-1190-40a0-83e1-ece622fa9c2e port 34bf5469-5bdd-4b70-b23f-5015d71654e9 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:06:04.083794 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'Port': ['ID=f0a1e941-ba29-43f9-8b64-db23166bf9ed,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:04.088304 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:04.117007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.359017 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.546481 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 7ade3842-cd45-419d-9045-8626ba1fdbe0 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:04.617234 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Allocated IP 172.20.0.1 (9a3ab75c-0613-43fc-b406-9209bc3e8724/0f050b21-5ec6-4623-a5c0-4e203399a27b/c366b56b-4d33-4772-8a44-91c29911caf1) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:04.617971 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.640536 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Transition to ACTIVE for port object 34bf5469-5bdd-4b70-b23f-5015d71654e9 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:04.648695 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 7ade3842-cd45-419d-9045-8626ba1fdbe0 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:04.649108 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.659343 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.721665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.725323 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-ef837e1d-feff-41f1-a8b0-8e87162e10d9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:04.794027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.815628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.817741 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Transition to ACTIVE for port object c366b56b-4d33-4772-8a44-91c29911caf1 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:04.897031 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Attempting to bind port 34bf5469-5bdd-4b70-b23f-5015d71654e9 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:04.897487 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Attempting to bind port 34bf5469-5bdd-4b70-b23f-5015d71654e9 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:04.897859 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Attempting to bind port 34bf5469-5bdd-4b70-b23f-5015d71654e9 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:04.899215 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.903647 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:04.905784 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Pushing event updated for resources: {'Port': ['ID=34bf5469-5bdd-4b70-b23f-5015d71654e9,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:04.911004 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.186s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:04.911888 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 5, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:04.912253 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:04.912981 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.913138 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:04.913551 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Bound port: 34bf5469-5bdd-4b70-b23f-5015d71654e9, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:04.913949 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:04.973548 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:04.976571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:04.979012 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:04.999433 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] For port 34bf5469-5bdd-4b70-b23f-5015d71654e9, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:05.004098 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] For port 34bf5469-5bdd-4b70-b23f-5015d71654e9, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=34bf5469-5bdd-4b70-b23f-5015d71654e9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:05.023146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Pushing event updated for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:05.039864 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:05.056290 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.056805 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.075854 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.079916 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:05.115310 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.123998 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6daecbbe-a1ee-4cc1-a72d-e53e4a6268d3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:05.160472 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:05.161185 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:05:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:05.256423 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Pushing event updated for resources: {'Port': ['ID=34bf5469-5bdd-4b70-b23f-5015d71654e9,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:05.259798 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.264369 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.184s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:05.327465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'Port': ['ID=f0a1e941-ba29-43f9-8b64-db23166bf9ed,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:05.330459 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e72faa36-046c-44dd-8570-08611ca71f00 None None] Provisioning for port c366b56b-4d33-4772-8a44-91c29911caf1 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:05.338879 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.215s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:05.358995 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e72faa36-046c-44dd-8570-08611ca71f00 None None] Provisioning complete for port c366b56b-4d33-4772-8a44-91c29911caf1 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:05.359485 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e72faa36-046c-44dd-8570-08611ca71f00 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.462615 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Floating IP 446b2090-daad-4112-afca-bdc8632398c4 associated. External IP: 172.27.0.64, port: fb9be3d2-4022-41e3-825a-ddf6079de0ba. May 24 10:06:05.477356 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:05.560147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e72faa36-046c-44dd-8570-08611ca71f00 None None] Port c366b56b-4d33-4772-8a44-91c29911caf1 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:06:05.615401 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.863315 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-27795f25-eea9-4ae7-a7b6-1ba734c560c7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.864273 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Router 7ead47d0-d22c-480f-8edb-a291a66ba849 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:06:05.910241 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.989812 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:05.994464 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:01Z', 'revision_number': 3, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:06.002575 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a8748b1-dfa5-4b8b-965a-e50919476385 None None] The sync data for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi is: [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:01Z', 'revision_number': 3, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:06.058190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.075832 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:06.084245 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2aba19f9-747a-4a0b-aea7-497b985ef53d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:06.087117 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Sync routers for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:06.102430 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:06.103489 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:06.173005 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Pushing event updated for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:06.175493 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.178747 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.179449 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:06.317628 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 3.1315751 May 24 10:06:06.319176 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:06:06.329931 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58592) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:06.339289 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-205366445', 'description': 'tempest-secgroup-smoke-205366445 description', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:06.356845 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:06.384442 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:06.385869 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:06.388943 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:06.400595 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:06:06.407754 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:06.575960 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 34bf5469-5bdd-4b70-b23f-5015d71654e9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:06.691917 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.756384 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:06.766765 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:06.770902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:06.783302 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:06.796585 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e9053dc4-5e94-45d0-a0f7-c728a6b13d05 None None] Provisioning complete for port c366b56b-4d33-4772-8a44-91c29911caf1 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:06.797469 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e9053dc4-5e94-45d0-a0f7-c728a6b13d05 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.835491 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d565b52e-b111-4808-86c2-487850223eb2 req-9e69d7f5-a070-4fae-bc74-3f4820cf672a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:06.844291 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-9e69d7f5-a070-4fae-bc74-3f4820cf672a service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1098 time: 0.3503253 May 24 10:06:06.902396 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e9053dc4-5e94-45d0-a0f7-c728a6b13d05 None None] Port c366b56b-4d33-4772-8a44-91c29911caf1 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:06:06.904632 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 34bf5469-5bdd-4b70-b23f-5015d71654e9 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:06.911985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 34bf5469-5bdd-4b70-b23f-5015d71654e9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:06.912362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:06.967743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Router 91d55ed4-1190-40a0-83e1-ece622fa9c2e has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:06:07.144139 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '446b2090-daad-4112-afca-bdc8632398c4', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'port_id': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 479, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'mac_address': 'fa:16:3e:e9:3d:dc', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:04Z', 'revision_number': 0, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:07.154675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:07.155932 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Checking router: 7ead47d0-d22c-480f-8edb-a291a66ba849 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:07.180276 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:07.180276 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:07.189653 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 34bf5469-5bdd-4b70-b23f-5015d71654e9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=34bf5469-5bdd-4b70-b23f-5015d71654e9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:07.214138 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:07.236609 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:07.241492 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2f2bcce7-5113-45f8-b1c1-aceb5e048932 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:07.292882 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d565b52e-b111-4808-86c2-487850223eb2 req-356f371a-b0f6-4102-9de3-e06ad716bfcb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:07.294967 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-356f371a-b0f6-4102-9de3-e06ad716bfcb service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.4297135 May 24 10:06:07.440283 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=34bf5469-5bdd-4b70-b23f-5015d71654e9,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:07.443216 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-6a62b215-f863-4723-b7e5-9f5cb4bd50f2 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 998 time: 0.1282208 May 24 10:06:07.451800 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:07.548968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:07.623329 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:07.634767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:07.651883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ceb75def-7be3-44cc-be41-2fddd3cb9076 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:07.669292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d565b52e-b111-4808-86c2-487850223eb2 req-c8c92da2-42a4-4f7f-80b6-b5f11eb0f1e4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:07.670750 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-c8c92da2-42a4-4f7f-80b6-b5f11eb0f1e4 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.1988595 May 24 10:06:07.686631 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroup': ['ID=74db5aed-164a-4d86-968e-bc83de180d04,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:07.873327 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.221s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:07.900950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:07.907514 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] 10.222.0.67 "PUT /v2.0/routers/91d55ed4-1190-40a0-83e1-ece622fa9c2e/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.3138106 May 24 10:06:07.928091 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d565b52e-b111-4808-86c2-487850223eb2 req-d64b9e36-44c1-4af3-8b20-5fbd56ed257a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:07.929804 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-d64b9e36-44c1-4af3-8b20-5fbd56ed257a service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2356665 May 24 10:06:08.042484 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.043926 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 1.7106416 May 24 10:06:08.049425 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58612) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:08.059525 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:08.070596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:08.078755 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:08.084529 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:08.106076 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:08.107245 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-786748b0-11b2-4a69-97f8-54791022f1d8 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1689186 May 24 10:06:08.215213 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.404421 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d565b52e-b111-4808-86c2-487850223eb2 req-704c2c2d-0e7c-4961-ba35-6f5eb444dbc7 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2871163 May 24 10:06:08.572910 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.632736 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '446b2090-daad-4112-afca-bdc8632398c4', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'port_id': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 479, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'mac_address': 'fa:16:3e:e9:3d:dc', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:04Z', 'revision_number': 0, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:08.651137 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 34bf5469-5bdd-4b70-b23f-5015d71654e9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:08.653597 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8360c8a-cab3-4f18-a67a-efe364ad5da3 None None] The sync data for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi is: [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '446b2090-daad-4112-afca-bdc8632398c4', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'floating_ip_address': '172.27.0.64', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'port_id': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 479, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'mac_address': 'fa:16:3e:e9:3d:dc', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:04Z', 'revision_number': 0, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:08.658336 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.661829 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eb4d2a2d-7953-4a81-9b31-583b68f88353 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:08.690869 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=3386ed6b-6bcb-48db-8204-ec847cadce6d,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:08.708308 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:08.836224 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.840326 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.7891479 May 24 10:06:08.847012 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58630) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:08.855053 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:08.863494 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:08.873683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 34bf5469-5bdd-4b70-b23f-5015d71654e9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:08.873683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:08.874320 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:08.877312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:08.888795 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:08.897460 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Sync routers for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:08.932030 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:08.932685 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:08.999368 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.053606 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.060830 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dfb438df-f3ee-4a69-b2d0-63cbce85fbd5 None None] New status for floating IP 446b2090-daad-4112-afca-bdc8632398c4: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:06:09.195256 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.235505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.239628 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:09.260562 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:09.271898 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=1c5f74e6-e993-462e-9433-e262c87423b4,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:09.275518 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:09.342988 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.344206 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.4969816 May 24 10:06:09.354005 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58640) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:09.360489 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:09.369044 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:09.374943 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:09.379749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:09.388302 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:09.397363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58644) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:09.409833 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1246651160-network', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:09.416886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:09.422886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Usage tracker for resource:network and tenant:11d14c1ea7714a25b90a7b8ea78444f3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:09.427783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:09.436732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:09.513410 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.561830 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Made reservation on behalf of 11d14c1ea7714a25b90a7b8ea78444f3 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:06:09.571902 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.604606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.633557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:09.642040 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Usage tracker for resource:security_group_rule and tenant:11d14c1ea7714a25b90a7b8ea78444f3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:09.648231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:09.660791 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:09.662822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.671663 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 2, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:09.676466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Checking router: 91d55ed4-1190-40a0-83e1-ece622fa9c2e for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:09.677168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Updating router 91d55ed4-1190-40a0-83e1-ece622fa9c2e port c366b56b-4d33-4772-8a44-91c29911caf1 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:06:09.734302 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.795160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.805187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.812593 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:09.843071 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=0137f162-7a3b-44a5-8207-bc69312db917,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:09.845781 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:09.983711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:09.986843 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3d05cc29-7758-4b23-8c04-2aa9f8618f84 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:10.020329 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ab9e58e0-9c36-4c8f-8dc8-3488ab897f48,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:10.023354 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 92} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:06:10.025246 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:10.034858 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.035448 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Added segment feab763c-9341-411a-b433-e4628ba3e968 of type vlan for network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 May 24 10:06:10.044942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.051368 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-40d58044-e1dc-4516-ab3f-590dd045aa2f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.6921885 May 24 10:06:10.053817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58654) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:10.061691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:10.065718 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.069894 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:10.073942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:10.077711 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:10.083855 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:10.157134 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.169196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Transition to ACTIVE for port object c366b56b-4d33-4772-8a44-91c29911caf1 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:10.202367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.204582 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:10.205695 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.213887 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2527f450-f576-4eba-973e-1ad4732f11e4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:10.214337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.268856 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.273610 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:10.342976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.406787 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Pushing event updated for resources: {'Network': ['ID=06601b3d-f3ef-4ad5-b8fc-080d71f88279,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:10.409646 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.196s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:10.412732 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.416675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-98526e29-1f27-42d1-b80d-391440f68761 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:10.430268 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:06:10.446547 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=cbf793ab-8b2e-4982-ada9-0be3fc1c64ed,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:10.451285 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:10.479304 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.0779109 May 24 10:06:10.479730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58662) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:10.487242 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Request body: {'subnet': {'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1246651160-subnet', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:10.494478 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd53d70c-023a-4eb4-8d16-f5fb47e23741 None None] Pushing event updated for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:10.495493 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:10.504786 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Usage tracker for resource:subnet and tenant:11d14c1ea7714a25b90a7b8ea78444f3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:10.505100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-fd53d70c-023a-4eb4-8d16-f5fb47e23741 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.231s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:10.510784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:10.523373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:10.589664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.593319 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-883a7fcd-b99b-4531-8b7d-5c35cf4186c9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.5353031 May 24 10:06:10.596458 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58668) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:10.602566 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:10.611263 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:10.615976 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:10.620253 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:10.628846 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:10.632455 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Attempting to bind port c366b56b-4d33-4772-8a44-91c29911caf1 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:10.635654 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Attempting to bind port c366b56b-4d33-4772-8a44-91c29911caf1 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:10.635654 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Attempting to bind port c366b56b-4d33-4772-8a44-91c29911caf1 on network 9a3ab75c-0613-43fc-b406-9209bc3e8724 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:10.639939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 6, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:10.640325 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Checking segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:10.640741 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.641186 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Bound using segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:10.641723 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Bound port: c366b56b-4d33-4772-8a44-91c29911caf1, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:10.642050 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.698198 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Made reservation on behalf of 11d14c1ea7714a25b90a7b8ea78444f3 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:06:10.701220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.741010 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.securitygroups_db [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Project 4b47c7cc67c545eab92ea86cde1abac7 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:06:10.753650 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.757288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] For port c366b56b-4d33-4772-8a44-91c29911caf1, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:10.761467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] For port c366b56b-4d33-4772-8a44-91c29911caf1, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c366b56b-4d33-4772-8a44-91c29911caf1,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:10.849727 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.858904 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:10.864408 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:10.944600 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:10.945266 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:10.993810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd53d70c-023a-4eb4-8d16-f5fb47e23741 None None] Pushing event updated for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:11.003021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-fd53d70c-023a-4eb4-8d16-f5fb47e23741 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:11.080571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.084929 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-29f131f4-b750-440f-a72b-3d950a446bc4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:11.110726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=81e2de9a-f3fd-410d-b8ad-d0355905b5cf,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:11.116999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:11.180424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:11.189168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.191196 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1345d300-c08e-4b29-89b8-4d908e954a4f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.5937207 May 24 10:06:11.194246 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58680) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:11.203265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'security_group_rule': {'security_group_id': '74db5aed-164a-4d86-968e-bc83de180d04', 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:11.213039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:11.219426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Usage tracker for resource:security_group_rule and tenant:4b47c7cc67c545eab92ea86cde1abac7 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:11.223749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:11.232456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:11.321113 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.securitygroups_db [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Project 4b47c7cc67c545eab92ea86cde1abac7 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:06:11.324460 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.330026 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.330897 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f90196ed-2510-4a6e-894f-7d7268f4edc6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:11.428196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.454713 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Pushing event updated for resources: {'Subnet': ['ID=fa4071c3-553b-41c3-bee4-024bec5e4c11,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:11.458851 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:11.459761 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Checking segment: {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:11.460147 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Network hostable DHCP agents. Network: 06601b3d-f3ef-4ad5-b8fc-080d71f88279, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:06:11.493759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.494902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f4e8b6cc-3612-4190-a551-06dcbea8e233 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:11.515594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=5e746289-5e9f-4f0f-b51e-809b1a7fd5c2,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:11.519825 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:11.571955 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:06:11.629117 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.633298 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-5a8285d3-10b8-427e-866a-0e951909ca87 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.4345553 May 24 10:06:11.659248 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 06601b3d-f3ef-4ad5-b8fc-080d71f88279) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:11.678437 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-941d3613-d5bb-46dd-ac0a-26b4ff4f5174 None None] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:06:11.686284 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:11.686897 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:11.778350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b1d5058e-32e8-4553-8174-6ede2d8454d4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:06:11.853730 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-bdfb1db6-53a0-45a1-973c-b33550a64b71 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.3717461 May 24 10:06:11.859166 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58694) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:11.868175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:10Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:11.870959 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b1d5058e-32e8-4553-8174-6ede2d8454d4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:06:11.873448 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b1d5058e-32e8-4553-8174-6ede2d8454d4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-205366445&fields=id&tenant_id=4b47c7cc67c545eab92ea86cde1abac7 HTTP/1.1" status: 200 len: 267 time: 0.1436777 May 24 10:06:11.873723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:11.877137 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73b9ed92-0cbf-4a8e-961f-2aed840312de None None] The sync data for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi is: [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:10Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:11.887939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:11.895498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Usage tracker for resource:router and tenant:11d14c1ea7714a25b90a7b8ea78444f3 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:11.900491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:11.921347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:11.941603 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'fixed_ips': [{'subnet_id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:06:11.942619 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.034959 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-71c05161-e181-438f-9375-547507dc16ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:12.036369 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.037270 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-71c05161-e181-438f-9375-547507dc16ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1579220 May 24 10:06:12.037855 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.057087 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.057654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'port': {'security_groups': ['e83316d5-9d97-4e86-bce6-f8bde9c85f77', '74db5aed-164a-4d86-968e-bc83de180d04']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:12.165358 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Allocated IP 172.20.0.2 (06601b3d-f3ef-4ad5-b8fc-080d71f88279/fa4071c3-553b-41c3-bee4-024bec5e4c11/8b325555-7840-441d-aa3f-d98efeaa4c4f) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:12.207862 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.402313 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.414070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Transition to ACTIVE for port object 8b325555-7840-441d-aa3f-d98efeaa4c4f will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:12.434839 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Transition to ACTIVE for port object 8b325555-7840-441d-aa3f-d98efeaa4c4f will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:12.527764 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.557350 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:12.625278 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:12.634301 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:12.638606 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:12.684548 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:12.687616 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.709677 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c366b56b-4d33-4772-8a44-91c29911caf1 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:12.778109 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Pushing event updated for resources: {'Port': ['ID=8b325555-7840-441d-aa3f-d98efeaa4c4f,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:12.780350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.782401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.225s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:12.892416 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.900345 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Attempting to bind port 8b325555-7840-441d-aa3f-d98efeaa4c4f on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:12.900959 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Attempting to bind port 8b325555-7840-441d-aa3f-d98efeaa4c4f by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:12.901439 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Attempting to bind port 8b325555-7840-441d-aa3f-d98efeaa4c4f on network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:12.911953 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['fb9be3d2-4022-41e3-825a-ddf6079de0ba'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:06:12.917105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:12.953127 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 6, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:12.953630 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Checking segment: {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:12.953985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:12.954401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Bound using segment: {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:12.954910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Bound port: 8b325555-7840-441d-aa3f-d98efeaa4c4f, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:12.955298 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.032450 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:13.033188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.036045 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0023a315-b3fd-4e0b-817f-ed798b0d1687 req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "PUT /v2.0/ports/fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 981 time: 0.9893670 May 24 10:06:13.036639 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:06:13.075965 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port c366b56b-4d33-4772-8a44-91c29911caf1 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:13.095552 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port c366b56b-4d33-4772-8a44-91c29911caf1 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:13.095939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.115982 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=6']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:13.118166 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f212a2e-6272-47b9-b3d4-b07b77f2bdae req-4ca318ac-7def-4b80-a93f-3ff666d58f9b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:13.126258 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f212a2e-6272-47b9-b3d4-b07b77f2bdae req-4ca318ac-7def-4b80-a93f-3ff666d58f9b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.2827771 May 24 10:06:13.147569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.226s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:13.155706 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['fb9be3d2-4022-41e3-825a-ddf6079de0ba'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:06:13.203974 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f212a2e-6272-47b9-b3d4-b07b77f2bdae req-83554f46-27da-4d79-a05c-7e1c60c5e4a9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0765719 May 24 10:06:13.371288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] For port 8b325555-7840-441d-aa3f-d98efeaa4c4f, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:13.411660 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] For port 8b325555-7840-441d-aa3f-d98efeaa4c4f, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8b325555-7840-441d-aa3f-d98efeaa4c4f,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:13.487582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6e850c5-d106-4272-a913-08fbc194390c req-2c5c9e4f-64b3-4ace-829b-0e1d07a82dfb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:13.492350 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6e850c5-d106-4272-a913-08fbc194390c req-2c5c9e4f-64b3-4ace-829b-0e1d07a82dfb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.2706292 May 24 10:06:13.504654 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.542896 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:13.589533 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6e850c5-d106-4272-a913-08fbc194390c req-35925cb0-c15a-42e8-962a-56f5ed04940a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0915258 May 24 10:06:13.593273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58720) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:13.677191 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.733176 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.769887 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-fdc348c0-3d6a-44b3-a2ab-fbf98d9e41bd None None] Provisioning complete for port fb9be3d2-4022-41e3-825a-ddf6079de0ba triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:13.770357 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fdc348c0-3d6a-44b3-a2ab-fbf98d9e41bd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.776480 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:06:13.777294 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.853715 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:13.900047 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:13.906153 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Usage tracker for resource:security_group_rule and tenant:ca4192b709c24b32965648a83b023209 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:13.911805 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:13.933538 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:13.934611 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Pushing event updated for resources: {'Port': ['ID=8b325555-7840-441d-aa3f-d98efeaa4c4f,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:13.958311 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Allocated IP 172.27.0.252 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/e2803d5b-53e2-4d18-9296-07a8b49dd7c9) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:13.963791 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.420s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:14.051144 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port c366b56b-4d33-4772-8a44-91c29911caf1, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c366b56b-4d33-4772-8a44-91c29911caf1,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:14.095202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.124866 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-b6d6fce2-4cb7-45d6-b61a-ca06acbd1d22 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:14.128026 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-b6d6fce2-4cb7-45d6-b61a-ca06acbd1d22 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1137 time: 0.3297167 May 24 10:06:14.162944 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.170594 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.176340 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:14.196832 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.236900 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-19b716b5-d05e-4afc-a793-a078fe77a3d9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:14.316197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.325412 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:14.398048 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:14.413775 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.237s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:14.453339 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ca4192b709c24b32965648a83b023209&name=default HTTP/1.1" status: 200 len: 2979 time: 0.8578069 May 24 10:06:14.464376 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7f245d97-8788-4f43-ac1c-27de2e02aaab,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:14.472539 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:14.473502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58732) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:14.482932 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.246s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:14.502393 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:14.556489 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-aed97443-0fa2-47a5-bea6-774cc225d9c4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:14.558987 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-aed97443-0fa2-47a5-bea6-774cc225d9c4 service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.4234807 May 24 10:06:14.605502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:14.688258 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Pushing event updated for resources: {'Port': ['ID=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:14.697546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.373s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:14.740331 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c366b56b-4d33-4772-8a44-91c29911caf1 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:14.832189 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-f04029db-72ad-4c8b-90d4-56202769f0a8 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 1000 time: 0.2643108 May 24 10:06:14.889820 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.918279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-b0bd8034-95ad-439e-b855-5bdbf5ef3863 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:14.919989 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-b0bd8034-95ad-439e-b855-5bdbf5ef3863 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.0778632 May 24 10:06:14.962793 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port c366b56b-4d33-4772-8a44-91c29911caf1 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:14.964332 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:14.981680 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55149a4d-95f9-418a-951d-275b3b271a4d req-5f648758-ec5d-4e11-9676-9f09f8e8c78e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:14.983419 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-55149a4d-95f9-418a-951d-275b3b271a4d req-5f648758-ec5d-4e11-9676-9f09f8e8c78e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.5088053 May 24 10:06:15.034074 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-ad37c102-a8ce-4f22-b4d4-1e4dff3439ea service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:15.036142 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-ad37c102-a8ce-4f22-b4d4-1e4dff3439ea service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1076851 May 24 10:06:15.065297 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.143700 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-55149a4d-95f9-418a-951d-275b3b271a4d req-645b672a-beca-4303-ba24-704c147de796 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1202085 May 24 10:06:15.225772 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8b325555-7840-441d-aa3f-d98efeaa4c4f up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:15.252431 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-dda35c85-b80c-4f78-af6a-dc6fcde80292 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2095473 May 24 10:06:15.310778 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.323250 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cfc64ca4-b152-4bd4-9754-f473dab6a2ec admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:15.350388 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-04032a28-78ae-4ff9-8eec-389306864be5 None None] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:06:15.357601 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7f245d97-8788-4f43-ac1c-27de2e02aaab,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:15.364523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:15.396816 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 8b325555-7840-441d-aa3f-d98efeaa4c4f completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:15.426138 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eeff821c-e59d-4565-8b68-70f760d1fcdd req-998ca28a-5b00-44fb-bcec-3a4acd9af636 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1630585 May 24 10:06:15.447618 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.448766 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/7f245d97-8788-4f43-ac1c-27de2e02aaab HTTP/1.1" status: 204 len: 168 time: 0.9691167 May 24 10:06:15.465217 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.631431 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Provisioning for port 8b325555-7840-441d-aa3f-d98efeaa4c4f completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:15.641389 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Provisioning complete for port 8b325555-7840-441d-aa3f-d98efeaa4c4f triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:15.642259 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.836657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.874657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] For port 8b325555-7840-441d-aa3f-d98efeaa4c4f, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8b325555-7840-441d-aa3f-d98efeaa4c4f,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:15.966935 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:15.970211 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:15.975558 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:16.054128 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Pushing event updated for resources: {'Port': ['ID=8b325555-7840-441d-aa3f-d98efeaa4c4f,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:16.057100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:16.093909 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e9750241-0bf7-446f-bd7b-c0c9fb129dd6 None None] Provisioning complete for port 8b325555-7840-441d-aa3f-d98efeaa4c4f triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:16.093909 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e9750241-0bf7-446f-bd7b-c0c9fb129dd6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:16.350320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Router 0d750bba-941f-408f-9a1d-724b07fb0aae is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:06:16.471059 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc1c3447-776a-4486-89f1-9d6fde702b1e req-2d42474b-af52-4f8a-b5e6-8b4a8e15ba01 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:16.472582 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dc1c3447-776a-4486-89f1-9d6fde702b1e req-2d42474b-af52-4f8a-b5e6-8b4a8e15ba01 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1843221 May 24 10:06:16.506600 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8b325555-7840-441d-aa3f-d98efeaa4c4f up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:16.600240 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 8b325555-7840-441d-aa3f-d98efeaa4c4f triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:16.600372 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:16.638248 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:16.655682 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Sync routers for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:16.659573 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dc1c3447-776a-4486-89f1-9d6fde702b1e req-1644a8d0-edb9-4f7f-aba8-b35eb2df48f0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1328936 May 24 10:06:16.685076 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:16.685748 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:16.766559 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:16.767808 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 4.9069054 May 24 10:06:16.770834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58760) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:17.000315 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:17.330781 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:15Z', 'revision_number': 3, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:17.336174 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Checking router: 0d750bba-941f-408f-9a1d-724b07fb0aae for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:17.336630 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Updating router 0d750bba-941f-408f-9a1d-724b07fb0aae port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:06:17.426891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:17.487942 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:06:17.725368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:17.926289 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:17.944455 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Transition to ACTIVE for port object e2803d5b-53e2-4d18-9296-07a8b49dd7c9 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:18.034923 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.039151 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-14180773-2133-444e-9902-0cc79bdf13ab None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:18.120176 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e28a9d60-006c-4bc4-9846-4717754271e3 req-6158e417-1eec-4ba2-9dab-c5f7be6b3641 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:18.122202 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e28a9d60-006c-4bc4-9846-4717754271e3 req-6158e417-1eec-4ba2-9dab-c5f7be6b3641 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1782703 May 24 10:06:18.122393 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Attempting to bind port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:18.124392 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Attempting to bind port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:18.124392 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Attempting to bind port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:18.129272 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56ef0b49-0db6-4bd3-b930-ca82b74bd362 None None] Pushing event updated for resources: {'Port': ['ID=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:18.132134 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56ef0b49-0db6-4bd3-b930-ca82b74bd362 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:18.135510 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 6, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:18.135897 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:18.136358 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.136751 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:18.138150 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Bound port: e2803d5b-53e2-4d18-9296-07a8b49dd7c9, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:18.138244 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.217545 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] For port e2803d5b-53e2-4d18-9296-07a8b49dd7c9, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:18.220986 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] For port e2803d5b-53e2-4d18-9296-07a8b49dd7c9, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:18.222885 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e28a9d60-006c-4bc4-9846-4717754271e3 req-40e3d139-153a-49ce-b0c2-3a91fe996317 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0932603 May 24 10:06:18.232981 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Allocated IP 172.20.0.1 (06601b3d-f3ef-4ad5-b8fc-080d71f88279/fa4071c3-553b-41c3-bee4-024bec5e4c11/c3854b9c-1314-4484-9d55-15d804931dda) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:06:18.320837 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.325147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:18.399999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:18.400651 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:18.449127 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.461665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56ef0b49-0db6-4bd3-b930-ca82b74bd362 None None] Pushing event updated for resources: {'Port': ['ID=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:18.472184 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56ef0b49-0db6-4bd3-b930-ca82b74bd362 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:18.483788 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Transition to ACTIVE for port object c3854b9c-1314-4484-9d55-15d804931dda will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:18.596811 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:18.600888 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:18.605272 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d74f377d-0eb5-443e-856c-844ed2e2b6ef service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:18.675477 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:18.678578 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:18.682254 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:06:18.716017 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Pushing event updated for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:18.735384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:18.751301 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.007506 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.027249 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:15Z', 'revision_number': 3, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:18Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:19.034615 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-984f1327-c64d-4452-9970-db92f7cb6909 None None] The sync data for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi is: [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:15Z', 'revision_number': 3, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:18Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:19.069059 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a0f050ee-3ffb-4ef1-b352-76883ff918e7 None None] Provisioning for port c3854b9c-1314-4484-9d55-15d804931dda completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:19.075548 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a0f050ee-3ffb-4ef1-b352-76883ff918e7 None None] Provisioning complete for port c3854b9c-1314-4484-9d55-15d804931dda triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:19.076341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a0f050ee-3ffb-4ef1-b352-76883ff918e7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.199333 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-a0f050ee-3ffb-4ef1-b352-76883ff918e7 None None] Port c3854b9c-1314-4484-9d55-15d804931dda cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:06:19.401252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.510297 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.551356 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:19.647511 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1884362e-259d-4080-bebe-f34979c1cd5d req-bd174c22-e2f3-4d91-8447-2c7be9fd4b89 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:19.649272 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1884362e-259d-4080-bebe-f34979c1cd5d req-bd174c22-e2f3-4d91-8447-2c7be9fd4b89 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.2334113 May 24 10:06:19.665922 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:19.696900 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Pushing event updated for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:19.700372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:19.823190 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1884362e-259d-4080-bebe-f34979c1cd5d req-96097c3b-2da5-4d6f-ad50-fba62b3b8ee3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1476786 May 24 10:06:19.876927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:06:19.878534 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:06:20.135807 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ad4eb05f-aa05-4f7c-8339-5f6fb008b43d None None] Provisioning complete for port c3854b9c-1314-4484-9d55-15d804931dda triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:20.137907 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ad4eb05f-aa05-4f7c-8339-5f6fb008b43d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:20.143939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Router 0d750bba-941f-408f-9a1d-724b07fb0aae has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:06:20.197330 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:20.202401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-ad4eb05f-aa05-4f7c-8339-5f6fb008b43d None None] Port c3854b9c-1314-4484-9d55-15d804931dda cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:06:20.454712 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:06:20.471206 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d933c190-cbea-4b49-a3bd-0bab925f3e44 admin admin] 10.222.0.67 "PUT /v2.0/routers/0d750bba-941f-408f-9a1d-724b07fb0aae/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.6968236 May 24 10:06:20.578439 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e2803d5b-53e2-4d18-9296-07a8b49dd7c9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:20.733967 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:20.742744 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:20.744984 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:20.927946 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:20.947247 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port e2803d5b-53e2-4d18-9296-07a8b49dd7c9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:20.974351 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Sync routers for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:06:21.000870 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.001608 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:21.002844 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-56ef0b49-0db6-4bd3-b930-ca82b74bd362 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:21.010444 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:21.146972 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:21.151271 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:21.257535 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dce9b7a-8173-44f8-9684-da93092944d1 req-1c678662-cfd8-421a-9b24-7e573f403f2e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:21.271067 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7dce9b7a-8173-44f8-9684-da93092944d1 req-1c678662-cfd8-421a-9b24-7e573f403f2e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1445432 May 24 10:06:21.327224 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7dce9b7a-8173-44f8-9684-da93092944d1 req-f5383107-8bba-4bdb-9e90-aedfaaf6ea87 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0554752 May 24 10:06:21.352484 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:21.508291 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58808) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:21.517431 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-222702199', 'description': 'tempest-secgroup-smoke-222702199 description', 'project_id': 'ce96e2adbf10489aae7640c84f398b29'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:21.526216 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:21.533845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:21.540491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:21.554661 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:21.664129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.692802 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:21.700908 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:21.706294 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:21.716145 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:21.724480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58814) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:21.772258 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:18Z', 'revision_number': 4, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c3854b9c-1314-4484-9d55-15d804931dda', 'name': '', 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'mac_address': 'fa:16:3e:d4:54:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_interface', 'standard_attr_id': 505, 'fixed_ips': [{'subnet_id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:17Z', 'updated_at': '2021-05-24T10:06:19Z', 'revision_number': 2, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'subnets': [{'id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:21.776874 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Checking router: 0d750bba-941f-408f-9a1d-724b07fb0aae for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:06:21.780063 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Updating router 0d750bba-941f-408f-9a1d-724b07fb0aae port c3854b9c-1314-4484-9d55-15d804931dda binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:06:21.835134 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.854479 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.923017 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.926270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:21.933202 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-01c39d27-cc6f-40fb-8341-a329aaa5a85b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:21.950387 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:21.958370 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Usage tracker for resource:security_group_rule and tenant:c5b8fa1b8c9949e0ba5ebd79c7d076df is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:21.963814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:21.975301 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:22.012117 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroup': ['ID=821159f9-8a27-4e94-8508-812d7562ee2c,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:22.016915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:22.093774 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.095981 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.5859542 May 24 10:06:22.103031 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58822) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:22.108398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:22.117511 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:22.127061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:22.132704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:22.145619 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:22.218859 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.243248 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.256442 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Transition to ACTIVE for port object c3854b9c-1314-4484-9d55-15d804931dda will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:06:22.275319 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.303098 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.312872 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7bebacac-a040-4b0e-9b25-2ea79c8067ea admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:22.344212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.358942 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c5b8fa1b8c9949e0ba5ebd79c7d076df&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6308966 May 24 10:06:22.360445 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.362096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58828) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:22.364843 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:22.371356 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=55bad724-464b-46c9-95eb-5eebb3a7da71,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:22.381478 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.382293 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.382992 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.383592 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.384539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.385020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.385523 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.386167 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.386838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.387621 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.388410 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.389013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.393916 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:22.438442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.452161 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:22.482282 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=41c50c7e-26b6-4fe9-abc8-b8d08815adff,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:22.509159 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:22.514994 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.567395 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.629080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e2803d5b-53e2-4d18-9296-07a8b49dd7c9 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:22.635708 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Pushing event updated for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:22.686249 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.686587 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.687121 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.687613 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.688099 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.688584 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.689090 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.689590 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.690078 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.690560 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.691038 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.691537 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:22.700604 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.336s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:22.731257 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Attempting to bind port c3854b9c-1314-4484-9d55-15d804931dda on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:06:22.731257 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Attempting to bind port c3854b9c-1314-4484-9d55-15d804931dda by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:06:22.731257 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Attempting to bind port c3854b9c-1314-4484-9d55-15d804931dda on network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:06:22.733534 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 6, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:06:22.734144 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Checking segment: {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:06:22.734716 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.735414 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Bound using segment: {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:06:22.736059 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Bound port: c3854b9c-1314-4484-9d55-15d804931dda, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'feab763c-9341-411a-b433-e4628ba3e968', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 92, 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:06:22.736679 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.767673 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:22.770023 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.784926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cae7a3c3-b22a-4f48-a03f-f8063652a7c3 req-f6fca080-ae69-4406-b1d9-350b9174aecb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:22.786501 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cae7a3c3-b22a-4f48-a03f-f8063652a7c3 req-f6fca080-ae69-4406-b1d9-350b9174aecb tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.2024558 May 24 10:06:22.795332 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.804275 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0a95f356-ba2b-4dc2-97ae-84abf9f9299c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.6959476 May 24 10:06:22.804275 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58832) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:22.804769 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:22.813827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:22.820607 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:22.824566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:22.832153 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:22.843197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.846756 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] For port c3854b9c-1314-4484-9d55-15d804931dda, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:06:22.850228 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] For port c3854b9c-1314-4484-9d55-15d804931dda, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c3854b9c-1314-4484-9d55-15d804931dda,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:06:22.851406 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cae7a3c3-b22a-4f48-a03f-f8063652a7c3 req-0db2f637-2aba-4ec5-9327-64ce458b501c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0581160 May 24 10:06:22.854774 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8a6ba3db-0a26-4986-9211-6d0fe783ca4e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:22.879847 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=55bad724-464b-46c9-95eb-5eebb3a7da71,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:22.879847 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:22.884195 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:22.885345 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/55bad724-464b-46c9-95eb-5eebb3a7da71 HTTP/1.1" status: 204 len: 168 time: 0.5216885 May 24 10:06:22.978214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.045025 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.076862 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.081761 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-7b3316be-d032-4349-9920-896bd2a545dd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:23.161441 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:06:23.162091 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:06:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:23.177949 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.181971 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b64d1fdc-8751-48f7-96b1-370bc33ff5ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:23.225773 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=23d41920-8567-496b-a9fe-103d6a08cf02,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:23.251464 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:23.254326 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Pushing event updated for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=4']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:23.258288 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-9555c876-d09b-4562-bc64-c2cd06b8f23f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:23.366444 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.367680 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.5674453 May 24 10:06:23.373411 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58842) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:23.379336 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:23.388900 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:23.393518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:23.408967 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:23.408967 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:23.439188 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:06:23.520281 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.661956 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.736916 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.739239 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:23.761304 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=38a6bf21-9589-4410-b09c-5d7c6ff5971a,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:23.765513 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:23.813258 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:18Z', 'revision_number': 4, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c3854b9c-1314-4484-9d55-15d804931dda', 'name': '', 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'mac_address': 'fa:16:3e:d4:54:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_interface', 'standard_attr_id': 505, 'fixed_ips': [{'subnet_id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:17Z', 'updated_at': '2021-05-24T10:06:22Z', 'revision_number': 4, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'subnets': [{'id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:06:23.817759 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-659b7cc6-6136-4094-95c4-dfbc5516fdc2 None None] The sync data for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi is: [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:06:18Z', 'revision_number': 4, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c3854b9c-1314-4484-9d55-15d804931dda', 'name': '', 'network_id': '06601b3d-f3ef-4ad5-b8fc-080d71f88279', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'mac_address': 'fa:16:3e:d4:54:2b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_interface', 'standard_attr_id': 505, 'fixed_ips': [{'subnet_id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:17Z', 'updated_at': '2021-05-24T10:06:22Z', 'revision_number': 4, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'subnets': [{'id': 'fa4071c3-553b-41c3-bee4-024bec5e4c11', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:06:23.893371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:23.903322 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58858) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:23.903830 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-661b0c52-b4de-44e4-964f-21f82fcbfe09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5288332 May 24 10:06:23.913296 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:23.924252 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:23.929194 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:23.932596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:23.939817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:24.099348 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.213474 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.223655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f484829-9e60-44f4-9f63-a04d5a36eadb req-73d697a2-005c-4033-899d-b3a87fdac040 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:24.225514 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7f484829-9e60-44f4-9f63-a04d5a36eadb req-73d697a2-005c-4033-899d-b3a87fdac040 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.2232022 May 24 10:06:24.281537 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7f484829-9e60-44f4-9f63-a04d5a36eadb req-86775ca3-dc7f-42f1-9151-2fbe7a43b472 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0490491 May 24 10:06:24.354678 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.356796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:24.384150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=224d517c-b7c9-4f2b-bc64-7b71c2ac157b,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:24.388284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:24.520265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.521931 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-3bc2a77f-a4e2-4de4-8b37-3fadc11f79cc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.6168814 May 24 10:06:24.524572 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 58878) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:24.532822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:24.542139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:24.547451 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:24.551995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:24.563304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:24.571467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c3854b9c-1314-4484-9d55-15d804931dda up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:24.656812 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.securitygroups_db [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Project ce96e2adbf10489aae7640c84f398b29 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:06:24.656812 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.782364 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.783451 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port c3854b9c-1314-4484-9d55-15d804931dda completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:06:24.791771 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port c3854b9c-1314-4484-9d55-15d804931dda triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:24.792240 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.881502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:24.885860 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-640112c3-2c1a-476c-8155-b44785148702 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:24.938670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=a6fed865-7251-4bdf-ae1b-eb61545e1ce4,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:24.954711 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:24.989020 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.023691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.025743 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c1d2341c-224b-4728-9f52-c50df8d726e3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.4999547 May 24 10:06:25.030493 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 58916) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:25.039892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'security_group_rule': {'security_group_id': '821159f9-8a27-4e94-8508-812d7562ee2c', 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:06:25.045576 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port c3854b9c-1314-4484-9d55-15d804931dda, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c3854b9c-1314-4484-9d55-15d804931dda,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:25.057163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:06:25.066819 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:security_group_rule and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:06:25.071329 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:06:25.087081 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:06:25.138736 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.146572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2a3710fd-152c-48ba-845b-09550f9cb7eb None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:25.176908 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.securitygroups_db [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Project ce96e2adbf10489aae7640c84f398b29 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:06:25.182085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.247038 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:25.253936 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:25.282850 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.394228 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.396607 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:25.419723 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=e0bd8e6c-02a8-4431-b27b-a31fbf210971,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:25.425148 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:25.499352 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.500954 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.4683769 May 24 10:06:25.508455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4df616f8-90c0-43dc-9198-590d31e1c3ca req-45c7b38a-8275-44f7-a2d9-0ed4058a7487 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:25.514116 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:06:25.515211 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4df616f8-90c0-43dc-9198-590d31e1c3ca req-45c7b38a-8275-44f7-a2d9-0ed4058a7487 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1094499 May 24 10:06:25.594545 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4df616f8-90c0-43dc-9198-590d31e1c3ca req-e24f52ee-877b-4866-9c88-8dd4c9d276dd tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0645649 May 24 10:06:25.731399 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:25.747770 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:06:25.814147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.834869 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:06:25.853198 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:25.955078 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:25.959867 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:26.031849 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=6']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:26.043284 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:26.054819 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['00cb560b-b02f-42f4-9ee8-09089f5ae590'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:06:26.505275 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c3854b9c-1314-4484-9d55-15d804931dda up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:26.610409 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port c3854b9c-1314-4484-9d55-15d804931dda triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:26.611041 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:26.926419 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-077cf1cd-c38e-4eb9-9378-d0a7f8f49456 req-ee4ec79a-dd67-4e91-9355-b822c226eae3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:26.928006 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-077cf1cd-c38e-4eb9-9378-d0a7f8f49456 req-ee4ec79a-dd67-4e91-9355-b822c226eae3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 945 time: 0.1041291 May 24 10:06:26.999679 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-077cf1cd-c38e-4eb9-9378-d0a7f8f49456 req-7e3afe83-c54b-4948-abe3-c3b9a1ef7351 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0636570 May 24 10:06:27.511836 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:06:27.604289 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:06:27.604766 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:27.794653 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:27.825898 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:06:27.838693 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:27.921140 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:27.924293 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7cbe1387-13ae-45f5-ae33-f8bb526e554b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:28.011588 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=7']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:28.016371 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:28.129842 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'code': 200} May 24 10:06:28.390735 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3b25e48-3cdb-42ff-8369-3b95ba2429d4 req-0d10cde7-b5f8-4585-86d2-f006c63ba600 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:28.393548 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f3b25e48-3cdb-42ff-8369-3b95ba2429d4 req-0d10cde7-b5f8-4585-86d2-f006c63ba600 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1676667 May 24 10:06:28.489375 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f3b25e48-3cdb-42ff-8369-3b95ba2429d4 req-8a396a41-ef09-42de-a94b-66aaa0a51c67 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0864713 May 24 10:06:29.913433 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-451d70a1-a1fe-4ec3-af03-b4a01f01f03e req-4348ba53-7f83-4c0d-9d0c-f2354165df62 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:29.929825 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-451d70a1-a1fe-4ec3-af03-b4a01f01f03e req-4348ba53-7f83-4c0d-9d0c-f2354165df62 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1638203 May 24 10:06:29.991820 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-451d70a1-a1fe-4ec3-af03-b4a01f01f03e req-761a0756-48e7-45a5-a129-6cc27ce2b40e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0691078 May 24 10:06:31.253700 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9abaf420-21ac-4ec1-9ffa-ad5c64dfbb51 req-b9645f79-8d60-42b4-8042-762ca52a19be tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:31.257898 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9abaf420-21ac-4ec1-9ffa-ad5c64dfbb51 req-b9645f79-8d60-42b4-8042-762ca52a19be tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1166019 May 24 10:06:31.348655 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9abaf420-21ac-4ec1-9ffa-ad5c64dfbb51 req-684c6d64-f3ab-44ca-ae1f-c4f2cb8a8d18 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0840628 May 24 10:06:32.666320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3981f7bc-1ba5-4f20-bcf9-c3e076b22fbb req-70b18e99-009e-4b06-8f61-4b5a50321fac tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:32.670449 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3981f7bc-1ba5-4f20-bcf9-c3e076b22fbb req-70b18e99-009e-4b06-8f61-4b5a50321fac tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1216824 May 24 10:06:32.804839 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3981f7bc-1ba5-4f20-bcf9-c3e076b22fbb req-e2e740b5-3fb8-4eb5-8018-f50449e8fbd0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0955048 May 24 10:06:33.366771 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-60636e7b-2721-43cc-8f11-6e78e47d827a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:34.239791 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbea98a7-cf3a-4066-8805-c974a205d80f req-51d3839b-28fd-4213-ae7a-b80047bba9f6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:34.241945 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dbea98a7-cf3a-4066-8805-c974a205d80f req-51d3839b-28fd-4213-ae7a-b80047bba9f6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.2248774 May 24 10:06:34.324287 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dbea98a7-cf3a-4066-8805-c974a205d80f req-9d1e6489-b582-4f49-88f0-d36f0f03bf18 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0715837 May 24 10:06:35.233739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-352010d8-79a2-4630-a9a9-09d225e4246f req-12f4029c-b655-4d49-b067-785d1335ffa4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.239869 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-352010d8-79a2-4630-a9a9-09d225e4246f req-12f4029c-b655-4d49-b067-785d1335ffa4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1313119 May 24 10:06:35.390006 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-352010d8-79a2-4630-a9a9-09d225e4246f req-1b8eb173-d38b-45f5-b550-d971218db51f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.1413627 May 24 10:06:35.405344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-90ecf422-8aa9-4f36-86c1-99881238eb3c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.445609 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-90ecf422-8aa9-4f36-86c1-99881238eb3c service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1137 time: 0.3415921 May 24 10:06:35.633449 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b40e7ae0-9edd-4940-b3bf-f6fe239c5744 req-f4d84ad1-6a91-4342-9e52-d8de01f9f91d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.635435 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b40e7ae0-9edd-4940-b3bf-f6fe239c5744 req-f4d84ad1-6a91-4342-9e52-d8de01f9f91d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1145537 May 24 10:06:35.637000 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-30aac2c3-e170-443f-8582-7d70f3c9a30b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.639643 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-30aac2c3-e170-443f-8582-7d70f3c9a30b service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.2133474 May 24 10:06:35.694898 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b40e7ae0-9edd-4940-b3bf-f6fe239c5744 req-983df832-0c98-45eb-8360-1868b10da996 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0509155 May 24 10:06:35.780674 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-f64870d6-c464-403b-a663-ff7e6ca499f7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 1000 time: 0.1334786 May 24 10:06:35.859469 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-6d78310f-dac2-467f-bbec-8691f7373dfd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.861093 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-6d78310f-dac2-467f-bbec-8691f7373dfd service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.0730927 May 24 10:06:35.970223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-7e87f6d2-9c43-4bdf-8335-55b5817a1a27 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:35.972278 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-7e87f6d2-9c43-4bdf-8335-55b5817a1a27 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1040752 May 24 10:06:35.984659 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-277da2fd-2f5a-4dcd-a331-b7221bdf90c6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:35.990794 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:36.212105 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-a13d0cf4-fc1e-4065-b799-a4ce11449344 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2317693 May 24 10:06:36.760399 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a6af3bd6-7773-46a3-9e9d-0ebf07c084ea req-efd83d08-2adc-49f0-b6f2-e26ffcdd416f service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5411901 May 24 10:06:36.788068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8baa1fd1-ca6a-46c2-b074-2e993eaf152c req-6c88e25b-b3f0-4af2-87ea-bdb670e689ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:36.790735 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8baa1fd1-ca6a-46c2-b074-2e993eaf152c req-6c88e25b-b3f0-4af2-87ea-bdb670e689ae tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.2547021 May 24 10:06:36.902020 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8baa1fd1-ca6a-46c2-b074-2e993eaf152c req-b0296d94-d127-48ae-aa9d-73ef86d45717 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.1041975 May 24 10:06:37.054427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0b7acb9-b1e2-4fbb-9dbb-9e9e0c821d1e req-392dafeb-3b92-458e-ba6e-2219aea7f2fc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:37.060152 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0b7acb9-b1e2-4fbb-9dbb-9e9e0c821d1e req-392dafeb-3b92-458e-ba6e-2219aea7f2fc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1931045 May 24 10:06:37.150803 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0b7acb9-b1e2-4fbb-9dbb-9e9e0c821d1e req-43bcbba8-9b06-44a8-b18f-5a9c50cb6834 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0847337 May 24 10:06:37.239850 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:38.218980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8287ed40-7f21-4fe4-a3f9-5feb7333512c req-302c9d41-a8f5-42b5-92f7-fb8668d7e541 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:38.222416 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8287ed40-7f21-4fe4-a3f9-5feb7333512c req-302c9d41-a8f5-42b5-92f7-fb8668d7e541 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1345460 May 24 10:06:38.326147 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8287ed40-7f21-4fe4-a3f9-5feb7333512c req-8f1f0ef3-bf7d-4f96-96d6-5a9e10b0939b tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0920055 May 24 10:06:38.536205 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-496a4978-11b6-42ad-8240-2ce02edd96a2 req-c747366f-dbd3-4dac-9deb-89a7cb3a9892 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:38.539131 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-496a4978-11b6-42ad-8240-2ce02edd96a2 req-c747366f-dbd3-4dac-9deb-89a7cb3a9892 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1468480 May 24 10:06:38.606032 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-496a4978-11b6-42ad-8240-2ce02edd96a2 req-540778e7-a1f4-4408-a5f0-3054a41d9ed6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0583141 May 24 10:06:38.795115 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:39.914271 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f52a6f5-325d-430a-bf8c-58430a55dbce req-a4ae9e6c-5ce2-4108-909e-b73d016098c7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:39.917255 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f52a6f5-325d-430a-bf8c-58430a55dbce req-a4ae9e6c-5ce2-4108-909e-b73d016098c7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1229670 May 24 10:06:39.947236 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73203b25-6283-4676-b681-194056e58148 req-2c98ff5d-4473-4e0b-b4ed-6ab769374641 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:39.955086 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-73203b25-6283-4676-b681-194056e58148 req-2c98ff5d-4473-4e0b-b4ed-6ab769374641 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1883261 May 24 10:06:40.017222 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7f52a6f5-325d-430a-bf8c-58430a55dbce req-1ac02f93-5cc0-4ba1-b728-cb9105c5ad19 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0888999 May 24 10:06:40.025214 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-73203b25-6283-4676-b681-194056e58148 req-e6b2cf4b-3f82-4453-9ed6-d78061635797 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0680494 May 24 10:06:41.321184 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-983b5ed6-ed87-4fd9-9d56-1097bc1aac0e req-93a16c28-ef7c-4456-95a8-2747d6645229 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:41.323433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-983b5ed6-ed87-4fd9-9d56-1097bc1aac0e req-93a16c28-ef7c-4456-95a8-2747d6645229 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1294806 May 24 10:06:41.359001 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53039de9-9212-4685-9b0f-5f761ff5a3ee req-5798e8ee-1e73-4708-ae93-9f3a2dc5a71d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:41.382498 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53039de9-9212-4685-9b0f-5f761ff5a3ee req-5798e8ee-1e73-4708-ae93-9f3a2dc5a71d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1881135 May 24 10:06:41.434350 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-983b5ed6-ed87-4fd9-9d56-1097bc1aac0e req-f68dd004-1ffe-4b99-8621-f7ae31d07458 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0959406 May 24 10:06:41.495620 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-53039de9-9212-4685-9b0f-5f761ff5a3ee req-5ebb36d0-2fa1-4871-af0b-0dc089df4c0e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0808685 May 24 10:06:42.569087 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-399d662b-d06b-40bc-8be8-3dc9ee227be1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:42.572856 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-399d662b-d06b-40bc-8be8-3dc9ee227be1 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1099 time: 0.1575687 May 24 10:06:42.779546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55fdeb82-da67-4cfc-805f-c28c2f61bb2f req-dbcc17a3-1344-4c38-97ce-3fd93e0265cc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:42.781837 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-55fdeb82-da67-4cfc-805f-c28c2f61bb2f req-dbcc17a3-1344-4c38-97ce-3fd93e0265cc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1227913 May 24 10:06:42.860682 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-55fdeb82-da67-4cfc-805f-c28c2f61bb2f req-150cd508-8ee9-4ae9-b205-0b4fa3a263cd tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0651562 May 24 10:06:42.894179 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-4d06d2bf-b8c2-4acf-8a39-222296b1de12 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:42.896114 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-4d06d2bf-b8c2-4acf-8a39-222296b1de12 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.3163884 May 24 10:06:43.001960 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72e1bfc0-faae-415d-826d-5f8197a1d936 req-5c5544a3-faa4-4491-9ae4-68a43d05ef67 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:43.003592 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-72e1bfc0-faae-415d-826d-5f8197a1d936 req-5c5544a3-faa4-4491-9ae4-68a43d05ef67 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.2603021 May 24 10:06:43.067505 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-068989c8-a892-4b7d-9c2d-82ede725f152 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 1001 time: 0.1625626 May 24 10:06:43.086792 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-72e1bfc0-faae-415d-826d-5f8197a1d936 req-cc80bc5e-b9b6-48a4-b773-ed1ff99cd4fb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0756567 May 24 10:06:43.173512 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-a86f5ab7-8210-4747-8be2-c78a52477b32 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:43.175037 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-a86f5ab7-8210-4747-8be2-c78a52477b32 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0938609 May 24 10:06:43.259577 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-a5bd9c20-b084-40f3-9be2-fae1a19437e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:43.261091 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-a5bd9c20-b084-40f3-9be2-fae1a19437e1 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0774913 May 24 10:06:43.409219 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-781d8a59-875d-4936-a1bd-68dbb4bbc05f service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1400411 May 24 10:06:43.612867 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-30dc2586-21f1-4b5a-857e-7e44849fb737 req-a70f26a5-871a-48d8-b145-55dcf8b93b9a service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1953995 May 24 10:06:44.141290 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11b9686c-b77d-4c16-be7f-2c4d8cdbe7f7 req-681af169-656a-422b-b555-59a5770f68f2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:44.143271 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-11b9686c-b77d-4c16-be7f-2c4d8cdbe7f7 req-681af169-656a-422b-b555-59a5770f68f2 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.0991499 May 24 10:06:44.195730 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-11b9686c-b77d-4c16-be7f-2c4d8cdbe7f7 req-59a5eb3f-8fb1-4157-bc09-e00b65f63dc3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0457199 May 24 10:06:44.390946 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ef2913f-3447-45ce-8e45-947382d5c8a5 req-a09e0a8f-4b3b-4f01-b0b4-55d11bc6c0a2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:44.392445 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1ef2913f-3447-45ce-8e45-947382d5c8a5 req-a09e0a8f-4b3b-4f01-b0b4-55d11bc6c0a2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0894184 May 24 10:06:44.450275 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1ef2913f-3447-45ce-8e45-947382d5c8a5 req-2180b84c-16b1-4b07-89db-73229906d25a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0513656 May 24 10:06:45.484599 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-062f49df-f16b-4cb1-868c-2ab0977347ce req-8bddf15e-0bb4-4a05-839a-e57def64bc70 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:45.488057 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062f49df-f16b-4cb1-868c-2ab0977347ce req-8bddf15e-0bb4-4a05-839a-e57def64bc70 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 946 time: 0.1055098 May 24 10:06:45.552218 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062f49df-f16b-4cb1-868c-2ab0977347ce req-13f71505-2355-4dbf-a3c8-108cfbef4f43 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0561733 May 24 10:06:45.627172 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:06:45.756141 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-81af68f9-77d9-4379-a149-a179ff045834 req-1311a4ec-5032-4a9b-bfd9-7441a55a817e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:45.762261 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-81af68f9-77d9-4379-a149-a179ff045834 req-1311a4ec-5032-4a9b-bfd9-7441a55a817e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1647604 May 24 10:06:45.763036 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:45.816670 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:06:45.828417 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:06:45.848950 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-81af68f9-77d9-4379-a149-a179ff045834 req-fae560be-a55f-4d44-9231-6107f7889ac7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0701146 May 24 10:06:45.911357 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'code': 200} May 24 10:06:45.915728 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:45.919893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:06:46.002379 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=8']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:06:46.010908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.011280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.012266 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.012612 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.012946 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.013304 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.013631 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.013957 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.014289 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.014645 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.014964 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.015283 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:06:46.019350 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:06:46.873275 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7705e0aa-b076-49dd-be0c-e0856035a62f req-29ddcd40-6852-4167-b53e-9c245057143d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:46.874860 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7705e0aa-b076-49dd-be0c-e0856035a62f req-29ddcd40-6852-4167-b53e-9c245057143d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1709383 May 24 10:06:46.938494 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7705e0aa-b076-49dd-be0c-e0856035a62f req-c55ca074-70c0-472f-acda-5110ea41f43c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0555632 May 24 10:06:47.230950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdd70ebc-c375-45a6-b299-cd5b74cae071 req-80835316-1f2a-46a0-8fa8-233b468c6c27 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:47.232402 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdd70ebc-c375-45a6-b299-cd5b74cae071 req-80835316-1f2a-46a0-8fa8-233b468c6c27 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.2273936 May 24 10:06:47.282678 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdd70ebc-c375-45a6-b299-cd5b74cae071 req-99e2eb22-692a-4824-8184-654721920cfb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0431218 May 24 10:06:48.523274 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d3916d06-c2eb-4b67-a47e-a05dac01f5d3 req-9982eec6-95ce-456c-8a55-af90e59d367d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:48.525277 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d3916d06-c2eb-4b67-a47e-a05dac01f5d3 req-9982eec6-95ce-456c-8a55-af90e59d367d tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1107571 May 24 10:06:48.584170 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d3916d06-c2eb-4b67-a47e-a05dac01f5d3 req-38f2ebe7-3155-461e-9c1c-7847d0abc49c tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0482104 May 24 10:06:49.626414 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:06:50.177485 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:06:50.237405 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7129e9be-7074-4ad5-b351-f4fac0bea312 req-2d7d3d42-c230-4e7c-9ad1-fc58c187edb4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:50.237405 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7129e9be-7074-4ad5-b351-f4fac0bea312 req-2d7d3d42-c230-4e7c-9ad1-fc58c187edb4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0922258 May 24 10:06:50.289627 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7129e9be-7074-4ad5-b351-f4fac0bea312 req-29d9ff02-b830-4e4c-8c6e-ba0225cb872f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0463886 May 24 10:06:50.572360 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-44d9d827-0523-4ab2-9ce0-be880ee4c280 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:50.573732 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-44d9d827-0523-4ab2-9ce0-be880ee4c280 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1137 time: 0.1017799 May 24 10:06:50.749857 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-6aece96c-d049-4818-b81d-caa7a4707c83 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:50.751385 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-6aece96c-d049-4818-b81d-caa7a4707c83 service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.1549916 May 24 10:06:50.948806 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-ea48ec47-5030-4b90-ab48-45b3533ac92e service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 1000 time: 0.1894710 May 24 10:06:51.084061 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-31c10132-f2a4-4dcf-a459-0ff7dd54a610 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:51.087087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-31c10132-f2a4-4dcf-a459-0ff7dd54a610 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.1318791 May 24 10:06:51.100694 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b085a6f6-d412-4ba4-ad49-57e355b4b637 req-17bdf83f-702d-4d50-8e54-9217289d4cdc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:51.102537 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b085a6f6-d412-4ba4-ad49-57e355b4b637 req-17bdf83f-702d-4d50-8e54-9217289d4cdc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.2805758 May 24 10:06:51.211794 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b085a6f6-d412-4ba4-ad49-57e355b4b637 req-86d84155-4625-4774-ad83-51612656feea tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0750053 May 24 10:06:51.230961 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-87c69dd3-ef85-40bc-8a0b-20bec1f39522 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:51.233171 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-87c69dd3-ef85-40bc-8a0b-20bec1f39522 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1295598 May 24 10:06:51.414371 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-2e501bf9-d128-41bc-bc5c-34c42630c162 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1735618 May 24 10:06:51.429462 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59320) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:06:51.575575 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b9bc036-642f-46a1-9d95-01007dafe521 req-879fcc92-b75b-474f-80c9-eb9f7e877ee4 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1536210 May 24 10:06:51.669378 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-9ef2af52-3b6e-4816-889d-95c74ed852e3 service neutron] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.2365096 May 24 10:06:51.862287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b25a84c9-1d19-45b2-8fc3-42b90c9feff5 req-c28d84b7-7fcc-4d60-8068-7ea1cfd7ca83 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:51.864549 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b25a84c9-1d19-45b2-8fc3-42b90c9feff5 req-c28d84b7-7fcc-4d60-8068-7ea1cfd7ca83 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.1047606 May 24 10:06:51.930002 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b25a84c9-1d19-45b2-8fc3-42b90c9feff5 req-f12e50ed-c0be-4135-8e3c-ff1cda008d52 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0513244 May 24 10:06:52.429732 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-221a0340-8ad9-424f-82b8-ca76bb24162f req-7fee6039-e97a-4f30-9d83-685ac03d5ca9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:52.431409 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-221a0340-8ad9-424f-82b8-ca76bb24162f req-7fee6039-e97a-4f30-9d83-685ac03d5ca9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.0858400 May 24 10:06:52.482372 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-221a0340-8ad9-424f-82b8-ca76bb24162f req-a31a10bd-288d-4cc9-8c75-84689b66980a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0443254 May 24 10:06:53.130365 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c5736c5-8d62-45df-8154-907d45cbf7ec req-664c1a09-198d-4815-899a-cf2015e08380 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:53.131932 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2c5736c5-8d62-45df-8154-907d45cbf7ec req-664c1a09-198d-4815-899a-cf2015e08380 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0849676 May 24 10:06:53.188484 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2c5736c5-8d62-45df-8154-907d45cbf7ec req-fae34fc1-3a26-4005-a2bb-9079b22aff4a tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0480900 May 24 10:06:53.694422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05d81151-ec45-47eb-b96d-411622138233 req-9789471d-b9e7-4947-b18c-f45112f81a61 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:53.696076 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05d81151-ec45-47eb-b96d-411622138233 req-9789471d-b9e7-4947-b18c-f45112f81a61 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.0925019 May 24 10:06:53.747121 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05d81151-ec45-47eb-b96d-411622138233 req-cbe4f8c1-0836-4e2e-a44a-7840c382b193 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0434058 May 24 10:06:54.384434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c850233-f154-42f0-b52a-9538ed0e3280 req-e66414ea-130e-4634-a253-09ac6beb7e15 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:54.385994 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8c850233-f154-42f0-b52a-9538ed0e3280 req-e66414ea-130e-4634-a253-09ac6beb7e15 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0855129 May 24 10:06:54.440707 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8c850233-f154-42f0-b52a-9538ed0e3280 req-410c9c57-e583-4d0f-b21e-6437c364b1f5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0483775 May 24 10:06:54.488092 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:06:55.023654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9003e2e2-f9d6-467a-9977-fe1731b794f9 req-5ac30faf-a668-4db2-9783-776e21df5d9d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:55.026141 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9003e2e2-f9d6-467a-9977-fe1731b794f9 req-5ac30faf-a668-4db2-9783-776e21df5d9d tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1045394 May 24 10:06:55.086527 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9003e2e2-f9d6-467a-9977-fe1731b794f9 req-8793c80c-26e9-489c-b988-ec19d45baed4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0485990 May 24 10:06:55.662143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-365c3dd4-1799-4116-aeb0-9e08c53538f9 req-cf028d6b-3dd9-4a36-aa4a-83726b31abd5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:55.663715 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-365c3dd4-1799-4116-aeb0-9e08c53538f9 req-cf028d6b-3dd9-4a36-aa4a-83726b31abd5 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0984020 May 24 10:06:55.729699 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-365c3dd4-1799-4116-aeb0-9e08c53538f9 req-18d9633e-d73d-4cf5-8491-e4468dd5ba84 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0588005 May 24 10:06:56.440064 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77902fae-e948-4b7b-9a48-9b1cad09be6e req-16522a01-be59-495b-b32c-bc5363ee40ca tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:56.442613 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-77902fae-e948-4b7b-9a48-9b1cad09be6e req-16522a01-be59-495b-b32c-bc5363ee40ca tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.2131608 May 24 10:06:56.507864 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-77902fae-e948-4b7b-9a48-9b1cad09be6e req-ce4553a7-b16a-4446-ae7f-053246496ee7 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0558705 May 24 10:06:56.510445 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6635dafa-4e42-4b4a-9410-745311868e2d req-1c94cc32-3233-4783-b311-ad8b074bebd7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:56.513221 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-1c94cc32-3233-4783-b311-ad8b074bebd7 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec&tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb HTTP/1.1" status: 200 len: 1097 time: 0.1955628 May 24 10:06:56.643126 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6635dafa-4e42-4b4a-9410-745311868e2d req-1949397e-1983-44c7-a356-103dbfe83876 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:56.644632 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-1949397e-1983-44c7-a356-103dbfe83876 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1097 time: 0.1040211 May 24 10:06:56.936293 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-846955a0-a650-475b-913f-d2c162bf2fd1 req-e4169c77-d248-47c3-b3b2-0aec8e4423ac tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:56.937805 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-846955a0-a650-475b-913f-d2c162bf2fd1 req-e4169c77-d248-47c3-b3b2-0aec8e4423ac tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.0873222 May 24 10:06:56.987839 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6635dafa-4e42-4b4a-9410-745311868e2d req-d8540fb0-1473-481c-ad5c-b525d33fb1e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:56.989444 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-d8540fb0-1473-481c-ad5c-b525d33fb1e5 service neutron] 10.222.0.95 "GET /v2.0/networks?id=c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 200 len: 917 time: 0.3384233 May 24 10:06:57.000332 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-846955a0-a650-475b-913f-d2c162bf2fd1 req-36c7300e-d55f-4a73-a3b8-3832b120a2e9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&id=74db5aed-164a-4d86-968e-bc83de180d04&fields=id&fields=name HTTP/1.1" status: 200 len: 379 time: 0.0528731 May 24 10:06:57.119777 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-8512cc7c-c1e7-4a13-b7f2-22134f58bf99 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 200 len: 999 time: 0.1233499 May 24 10:06:57.195984 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6635dafa-4e42-4b4a-9410-745311868e2d req-04f39bbe-e551-413e-86b9-d0df6814447c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:57.197438 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-04f39bbe-e551-413e-86b9-d0df6814447c service neutron] 10.222.0.95 "GET /v2.0/subnets?id=327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 200 len: 856 time: 0.0654080 May 24 10:06:57.299754 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6635dafa-4e42-4b4a-9410-745311868e2d req-ad5197be-8b29-4386-a568-1007c331d5e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:57.301354 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-ad5197be-8b29-4386-a568-1007c331d5e5 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=c1fb75f5-deee-4236-9b04-1b6b5e617606&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0934412 May 24 10:06:57.550127 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-a9be30dc-6f5a-4d25-94f1-c774a78dc2d2 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2416439 May 24 10:06:57.767558 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6635dafa-4e42-4b4a-9410-745311868e2d req-6aaaaa38-1f66-402a-bd63-76635ec492d4 service neutron] 10.222.0.95 "GET /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2091570 May 24 10:06:57.800650 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9bbe7f0b-1698-4684-9092-1c5f7be61fe8 req-0398b4ff-bc4b-4625-9ce2-03235af9f1ce tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:57.814367 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9bbe7f0b-1698-4684-9092-1c5f7be61fe8 req-0398b4ff-bc4b-4625-9ce2-03235af9f1ce tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1562793 May 24 10:06:57.878294 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9bbe7f0b-1698-4684-9092-1c5f7be61fe8 req-cf3dc5fd-5fca-4e6e-a143-07344fd85316 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0624847 May 24 10:06:58.975782 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-0e32c59a-1a4e-4c3b-a5db-2dd608f1430f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:06:59.062544 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-0e32c59a-1a4e-4c3b-a5db-2dd608f1430f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:06:59.063765 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-0e32c59a-1a4e-4c3b-a5db-2dd608f1430f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-222702199&fields=id&tenant_id=ce96e2adbf10489aae7640c84f398b29 HTTP/1.1" status: 200 len: 267 time: 0.1579845 May 24 10:06:59.163450 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b52288c2-b7bf-4764-8f84-ac9b009fda05 req-fe80074a-a36e-4bf8-83d4-ab1b8cf63bbf tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:59.168753 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b52288c2-b7bf-4764-8f84-ac9b009fda05 req-fe80074a-a36e-4bf8-83d4-ab1b8cf63bbf tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1403780 May 24 10:06:59.243487 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b52288c2-b7bf-4764-8f84-ac9b009fda05 req-1e49adf4-1be3-4faa-87d0-8d55fd23b274 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0612869 May 24 10:06:59.325989 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-ea12b107-e2ba-415c-a5c1-6fb7aa65c8ce tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:06:59.327556 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-ea12b107-e2ba-415c-a5c1-6fb7aa65c8ce tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:06:59.411651 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-ea12b107-e2ba-415c-a5c1-6fb7aa65c8ce tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:06:59.413041 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-ea12b107-e2ba-415c-a5c1-6fb7aa65c8ce tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 809 time: 0.3262579 May 24 10:06:59.433350 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-778286fe-c475-4ea8-8a84-aff252c1cc84 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/quotas/ce96e2adbf10489aae7640c84f398b29 HTTP/1.1" status: 200 len: 375 time: 0.0105188 May 24 10:06:59.557914 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-406107a2-0bf4-42bc-b5a9-86906f5ffb11 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1176643 May 24 10:06:59.746108 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-9b80e3c8-0e6d-4a37-8b6f-57a584f89a27 service neutron] 10.222.0.67 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1739376 May 24 10:06:59.924279 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-bd3f9119-2a1b-4578-8df3-a5ca4b1b0bd5 service neutron] 10.222.0.67 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1698296 May 24 10:07:00.520134 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a05323c-f4b1-41ed-93b2-98b4709b621a req-9dabbe45-8c7b-459a-a0fc-59866e023d3a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:00.520134 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a05323c-f4b1-41ed-93b2-98b4709b621a req-9dabbe45-8c7b-459a-a0fc-59866e023d3a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1226428 May 24 10:07:00.584757 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a05323c-f4b1-41ed-93b2-98b4709b621a req-13e58c14-265f-4059-afe5-e88e9c90225e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0606465 May 24 10:07:00.873620 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-4e7b77ef-bddb-4644-97a8-6f4a4fbcfbfe req-70593a5d-81bd-40d8-9278-9a0a4cddbcb1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:07:00.977350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-4e7b77ef-bddb-4644-97a8-6f4a4fbcfbfe req-70593a5d-81bd-40d8-9278-9a0a4cddbcb1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:07:00.985876 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e7b77ef-bddb-4644-97a8-6f4a4fbcfbfe req-70593a5d-81bd-40d8-9278-9a0a4cddbcb1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.1698232 May 24 10:07:01.860950 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ba0330f-eb91-4963-8cd3-f050c6127ae1 req-d3eb12ef-aa9d-4b42-aa36-dbf587906277 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:01.868812 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7ba0330f-eb91-4963-8cd3-f050c6127ae1 req-d3eb12ef-aa9d-4b42-aa36-dbf587906277 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1095254 May 24 10:07:01.939269 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7ba0330f-eb91-4963-8cd3-f050c6127ae1 req-fb1546ff-5d4a-4b4c-82c6-2865c6b872b8 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0688083 May 24 10:07:02.113707 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-827990bf-48fd-4827-b9d4-9cbd966d3382 req-e733f373-4571-47fe-aad6-f115e7684635 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0575497 May 24 10:07:03.210515 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-abe343d8-4c35-4591-8814-5a0a8a5f821b req-079ce3d3-3d5a-4b78-998d-9003b6787079 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:03.223337 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-abe343d8-4c35-4591-8814-5a0a8a5f821b req-079ce3d3-3d5a-4b78-998d-9003b6787079 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1333730 May 24 10:07:03.334003 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-abe343d8-4c35-4591-8814-5a0a8a5f821b req-ddba1a19-b760-48d8-aafd-540e075c8212 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0989196 May 24 10:07:03.489034 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-42d8cdd4-9223-4590-8966-cad02f0ddd1b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:03.491278 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c0217edd-897c-46df-aeb4-1e2e5ddbc416 req-c7347d73-a002-4c5c-a5c6-1f925d09d454 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0616543 May 24 10:07:03.492023 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59484) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:03.709313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:03.748487 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:03.755840 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Usage tracker for resource:security_group_rule and tenant:3290e03d32d741e898855125b123e23d is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:07:03.759333 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:07:03.773190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:03.912051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:03.999633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:04.010976 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-acb1ceb4-db4b-4750-bcf1-bdead977c849 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:04.069637 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b05a5a5f-b00c-4bce-9f75-20a31686e4b9,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:04.076564 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=3290e03d32d741e898855125b123e23d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6006947 May 24 10:07:04.081914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59492) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:04.083384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.083947 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.084391 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.084834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.085345 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.085825 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.086222 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.086797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.087172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.087532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.088036 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.088403 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:04.094038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:04.173696 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:04.204533 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:04.348935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:04.353011 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9fd4d029-c979-4015-8fac-3e549b8acda4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:04.365789 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b05a5a5f-b00c-4bce-9f75-20a31686e4b9,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:04.369964 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:04.373671 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:04.374818 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d94493bd-e799-46d0-a4e4-08ea221c50a5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b05a5a5f-b00c-4bce-9f75-20a31686e4b9 HTTP/1.1" status: 204 len: 168 time: 0.2897065 May 24 10:07:04.657326 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-735f134f-c6e7-4729-b88b-846cebb4e32f req-d5272cbc-ef4a-4900-b471-b86dcb2e7734 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:04.673160 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735f134f-c6e7-4729-b88b-846cebb4e32f req-d5272cbc-ef4a-4900-b471-b86dcb2e7734 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1795728 May 24 10:07:04.808113 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-735f134f-c6e7-4729-b88b-846cebb4e32f req-33f8f8a1-4e2b-4625-9f2c-26fe13f2eea3 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1223474 May 24 10:07:04.808424 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-10147a78-f18f-4e10-bd5b-c11ee26d7f66 req-d3279c0a-d0a6-4317-8111-b7cb3c129111 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.1031733 May 24 10:07:05.868211 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b28d5cd4-40f6-4eac-863e-8c1083907c13 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:05.902773 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:06.085566 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3f0ff5aa-635f-441f-8049-56547bb10f52 req-e96f4dfd-30e9-4f69-9520-1482b50080a7 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.1009846 May 24 10:07:06.331877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a2fbd1c-d00c-41ea-bb26-c6489cd62b47 req-4d5feffc-b040-48dc-9b20-0c667315c0b5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:06.337399 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4a2fbd1c-d00c-41ea-bb26-c6489cd62b47 req-4d5feffc-b040-48dc-9b20-0c667315c0b5 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.3073697 May 24 10:07:06.415086 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4a2fbd1c-d00c-41ea-bb26-c6489cd62b47 req-a3a7ca55-3ee8-41cc-995d-72c9dfb65d62 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0725446 May 24 10:07:07.247807 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e8843e17-4cca-4bb1-9e90-090fb9249781 req-d8c9182e-84bd-48c7-8bc6-44c4ff5712c6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0535779 May 24 10:07:07.256118 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:07.661711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0431b5b0-ebb3-4dde-90e7-7836766ce649 req-676216d4-cb3a-4389-965b-dddc8e7ea175 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:07.663475 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0431b5b0-ebb3-4dde-90e7-7836766ce649 req-676216d4-cb3a-4389-965b-dddc8e7ea175 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1106691 May 24 10:07:07.728205 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0431b5b0-ebb3-4dde-90e7-7836766ce649 req-b930737b-ae00-4f11-b0a7-ee1678292985 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0549135 May 24 10:07:08.561354 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f1a7e025-5f87-4a64-a365-719cc260fc51 req-fefe157f-1aa6-4f0c-9762-785c7f7f08ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.1258404 May 24 10:07:08.754726 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:08.977442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0798a4de-37a4-4591-8a89-3239dd8a5cd0 req-da7586f5-5dc1-4475-b7e8-6e51032d84a4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:08.979183 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0798a4de-37a4-4591-8a89-3239dd8a5cd0 req-da7586f5-5dc1-4475-b7e8-6e51032d84a4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.0962634 May 24 10:07:09.044228 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0798a4de-37a4-4591-8a89-3239dd8a5cd0 req-4b837c06-be8d-41bf-84bd-a3bba12707df tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0563159 May 24 10:07:09.709555 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1aa6e92f-cbde-4406-9522-f4bcd8a3d0e4 req-aeaebdfe-4cef-46a2-b8b0-5e9b1bb055d5 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0476840 May 24 10:07:10.330065 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfdd6184-1f5b-4ee7-91e8-9f20a0d8f746 req-1a02b0ed-1aa8-431a-8767-2c2d016d7964 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:10.331930 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cfdd6184-1f5b-4ee7-91e8-9f20a0d8f746 req-1a02b0ed-1aa8-431a-8767-2c2d016d7964 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1560390 May 24 10:07:10.387487 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cfdd6184-1f5b-4ee7-91e8-9f20a0d8f746 req-5b434d29-a3ca-4f12-82c1-e75f569e53dc tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0490620 May 24 10:07:10.879777 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05bfb032-457c-450e-a3bf-41d35cbb908f req-913d2d25-d24f-4430-9044-3d2b4cd00273 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0484564 May 24 10:07:11.643434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61c75bf2-90fb-49c9-ad28-74b960707134 req-30427035-e520-4a40-96d8-e296ccc91990 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:11.645044 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-61c75bf2-90fb-49c9-ad28-74b960707134 req-30427035-e520-4a40-96d8-e296ccc91990 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.1044068 May 24 10:07:11.698871 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-61c75bf2-90fb-49c9-ad28-74b960707134 req-d8e5f810-73af-486b-97cc-9c338379489b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0453074 May 24 10:07:12.071466 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-71461933-6486-421e-9e0d-8be898641e5c req-2364cf55-31f8-46a7-98c4-83ecf2dcb6cb tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0510201 May 24 10:07:12.718210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:07:12.953913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:12.969748 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'port_id': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'mac_address': 'fa:16:3e:e3:79:aa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 90, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:07:12.996316 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c32894f-30b1-40e1-b0ab-a4b4eec3c054 req-34c30752-78db-47ba-be32-8f3fc46b9a98 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:12.998438 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3c32894f-30b1-40e1-b0ab-a4b4eec3c054 req-34c30752-78db-47ba-be32-8f3fc46b9a98 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 944 time: 0.0999842 May 24 10:07:13.035681 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:13.055191 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:07:13.057492 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3c32894f-30b1-40e1-b0ab-a4b4eec3c054 req-e9a9d500-bc5e-49e4-b15b-42d3defffe6a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0478420 May 24 10:07:13.072920 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:13.166231 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:13.168926 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:13.242725 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=9']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:13.247057 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:13.270144 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0591f837-f67b-421f-99a6-c262eb5104ac req-e619f753-8276-49ed-bfb7-bb3c149feee0 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0803182 May 24 10:07:13.989746 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['00cb560b-b02f-42f4-9ee8-09089f5ae590'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:07:14.034621 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-9d420056-739d-4dbe-b462-e5e7119aab0b tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:14.037453 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-9d420056-739d-4dbe-b462-e5e7119aab0b tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 809 time: 0.3044186 May 24 10:07:14.115513 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-f4a2307b-5adb-4dfd-b89f-68804593d101 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=ce96e2adbf10489aae7640c84f398b29 HTTP/1.1" status: 200 len: 379 time: 0.0691428 May 24 10:07:14.129971 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'port': {'device_id': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'admin_state_up': True, 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'security_groups': ['821159f9-8a27-4e94-8508-812d7562ee2c']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:07:14.464570 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3c55d784-8bd0-4a00-ad22-df412f8ffad4 req-b2f60d4b-810d-4068-8ccf-3722277c64ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0651913 May 24 10:07:14.523393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:14.547455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:14.574843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45240006-d8c1-4007-86df-7be767bddaf0 req-4f8fe23d-4b48-4c7e-a400-3ab1fb75a482 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:14.577780 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45240006-d8c1-4007-86df-7be767bddaf0 req-4f8fe23d-4b48-4c7e-a400-3ab1fb75a482 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 945 time: 0.2533927 May 24 10:07:14.656521 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45240006-d8c1-4007-86df-7be767bddaf0 req-3e777298-1238-4221-b529-2921236c9d4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0693014 May 24 10:07:14.697129 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Made reservation on behalf of ce96e2adbf10489aae7640c84f398b29 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:07:14.702179 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:14.921796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Allocated IP 172.20.0.9 (9a3ab75c-0613-43fc-b406-9209bc3e8724/0f050b21-5ec6-4623-a5c0-4e203399a27b/b5308ea7-0517-43e4-830b-7fd131b3a9c7) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:07:15.186803 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:15.215066 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Transition to ACTIVE for port object b5308ea7-0517-43e4-830b-7fd131b3a9c7 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:07:15.302557 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:15.308968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b0a0b490-d0e4-497e-9ed0-c5c5d2b04f8e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:15.385515 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:07:15.390089 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:07:15.394899 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:07:15.427517 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:15.434804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:15.484206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:15.485804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:15.667335 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.5463755 May 24 10:07:15.688437 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Request body: {'port': {'device_id': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:07:15.748262 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4291340d-fb63-4919-b935-54fee0c7ba88 None None] Provisioning for port b5308ea7-0517-43e4-830b-7fd131b3a9c7 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:07:15.755052 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-4291340d-fb63-4919-b935-54fee0c7ba88 None None] Provisioning complete for port b5308ea7-0517-43e4-830b-7fd131b3a9c7 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:07:15.755632 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4291340d-fb63-4919-b935-54fee0c7ba88 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:15.826726 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-4291340d-fb63-4919-b935-54fee0c7ba88 None None] Port b5308ea7-0517-43e4-830b-7fd131b3a9c7 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:07:15.854486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:15.856734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:07:15.872270 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52bbfc43-e8f0-4db1-9433-19e8308f922f req-6d8152fa-19c5-446a-bdd3-4974e0faafc6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:15.873982 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-52bbfc43-e8f0-4db1-9433-19e8308f922f req-6d8152fa-19c5-446a-bdd3-4974e0faafc6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:07:15.957366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 00cb560b-b02f-42f4-9ee8-09089f5ae590 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:07:15.957730 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:16.109695 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:16.118325 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-52bbfc43-e8f0-4db1-9433-19e8308f922f req-6d8152fa-19c5-446a-bdd3-4974e0faafc6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:07:16.126150 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-52bbfc43-e8f0-4db1-9433-19e8308f922f req-6d8152fa-19c5-446a-bdd3-4974e0faafc6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 931 time: 0.4972651 May 24 10:07:16.173729 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:16.186653 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:16.226666 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-52bbfc43-e8f0-4db1-9433-19e8308f922f req-91566bac-4546-47d3-a02f-915f611b6cd5 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0994239 May 24 10:07:16.302594 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:16.304569 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-fd53d70c-023a-4eb4-8d16-f5fb47e23741 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:16.391535 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=10']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:16.395385 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:16.472514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b5308ea7-0517-43e4-830b-7fd131b3a9c7 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:07:16.483668 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:16.506756 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Transition to ACTIVE for port object b5308ea7-0517-43e4-830b-7fd131b3a9c7 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:07:16.510175 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'code': 200} May 24 10:07:16.523789 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95bb5e41-c268-4dc3-806d-d25f05b56414 req-57b87eca-a719-4eb8-9dbb-b636d1d39f08 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:16.526621 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-95bb5e41-c268-4dc3-806d-d25f05b56414 req-57b87eca-a719-4eb8-9dbb-b636d1d39f08 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.1478224 May 24 10:07:16.602719 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-95bb5e41-c268-4dc3-806d-d25f05b56414 req-01ed7643-d9e1-4b77-a677-d3ee06c6a516 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0674112 May 24 10:07:16.628339 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:16.636225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:17.015126 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:17.022918 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.387s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:17.089494 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24bd6c95-fca5-4aa8-b1da-53370d22db17 req-37060cd1-736b-491b-be80-108349e6b427 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:17.091084 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-24bd6c95-fca5-4aa8-b1da-53370d22db17 req-37060cd1-736b-491b-be80-108349e6b427 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 1100 time: 0.1088769 May 24 10:07:17.107202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Attempting to bind port b5308ea7-0517-43e4-830b-7fd131b3a9c7 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:07:17.107635 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Attempting to bind port b5308ea7-0517-43e4-830b-7fd131b3a9c7 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:07:17.108078 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Attempting to bind port b5308ea7-0517-43e4-830b-7fd131b3a9c7 on network 9a3ab75c-0613-43fc-b406-9209bc3e8724 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:07:17.114315 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Port b5308ea7-0517-43e4-830b-7fd131b3a9c7 on network 9a3ab75c-0613-43fc-b406-9209bc3e8724 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:07:17.114699 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Attempting to bind port b5308ea7-0517-43e4-830b-7fd131b3a9c7 on network 9a3ab75c-0613-43fc-b406-9209bc3e8724 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:07:17.123028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 6, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:07:17.123561 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Checking segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:07:17.123945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Bound using segment: {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:07:17.124486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Bound port: b5308ea7-0517-43e4-830b-7fd131b3a9c7, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'c69a7001-1e8e-470a-bfb0-2489247a3dd6', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 81, 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:07:17.124907 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:17.139225 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-24bd6c95-fca5-4aa8-b1da-53370d22db17 req-078bfbba-ee7d-48ad-be0f-fab07a3eb53a service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0410655 May 24 10:07:17.228287 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:07:17.232593 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:07:17.332804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:17.341400 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:17.435918 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ba9347b-e9e3-4073-8580-68ebaea9e969 req-89438e24-596c-48b3-8f4b-19ebd3433be2 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:17.437801 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8ba9347b-e9e3-4073-8580-68ebaea9e969 req-89438e24-596c-48b3-8f4b-19ebd3433be2 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0888941 May 24 10:07:17.461631 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:17.463740 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:17.465170 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] 10.222.0.95 "PUT /v2.0/ports/b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 1093 time: 1.7842705 May 24 10:07:17.465973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:17.505681 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:17.506606 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8ba9347b-e9e3-4073-8580-68ebaea9e969 req-e1045afc-c5da-4cac-aff3-85c85bf983c7 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0625694 May 24 10:07:17.516863 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:17.729709 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'status': 'completed', 'code': 200} May 24 10:07:17.794415 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-397f110c-af18-4c9d-97f8-b729a35cf776 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:17.797014 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-397f110c-af18-4c9d-97f8-b729a35cf776 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1096 time: 0.1602519 May 24 10:07:17.874821 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ccc2575a-a811-4a46-a672-5f460e696464 req-35555866-1cc1-461b-be18-91b0d126cd42 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:17.876433 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ccc2575a-a811-4a46-a672-5f460e696464 req-35555866-1cc1-461b-be18-91b0d126cd42 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.1066735 May 24 10:07:17.931744 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ccc2575a-a811-4a46-a672-5f460e696464 req-2c208a2d-7df0-4908-8174-ad639b494474 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0486178 May 24 10:07:17.953453 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-5cdb6049-8119-41d4-80a8-f2b9b5087179 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 217 time: 0.1475554 May 24 10:07:18.044045 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-9c5d7655-fdea-4e52-a81b-864400326a13 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:18.045559 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-9c5d7655-fdea-4e52-a81b-864400326a13 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.0845578 May 24 10:07:18.203130 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-65961a1d-42f7-4ee8-93d3-eaba49680ef9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:18.204859 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-65961a1d-42f7-4ee8-93d3-eaba49680ef9 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1423225 May 24 10:07:18.277378 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59666) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:18.394036 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-2e441a21-1bde-4da5-81e0-615247ff889c service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1773696 May 24 10:07:18.450113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:18.478697 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:18.486218 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Usage tracker for resource:security_group_rule and tenant:462230f1837d49efa951a16d18453422 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:07:18.491509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:07:18.502435 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:18.569643 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2bf4c91a-70e7-4720-8510-468982cbc594 req-2dfff3d7-1444-41ba-bf30-9a52ea39cd07 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1691518 May 24 10:07:18.655229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:18.873209 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-97be32d9-8aa2-418b-aa89-05f411f3d657 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:18.874741 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-97be32d9-8aa2-418b-aa89-05f411f3d657 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1096 time: 0.1528347 May 24 10:07:18.886271 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c24d6eb5-5f72-44b2-8dcc-43fbd8be242b req-69d01aae-9ba2-41a9-90cf-685e78e31cd8 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:18.888455 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c24d6eb5-5f72-44b2-8dcc-43fbd8be242b req-69d01aae-9ba2-41a9-90cf-685e78e31cd8 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1283495 May 24 10:07:18.890983 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:18.898967 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-fcf27be0-dccf-4aae-9b6f-66d7ff933ed3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:18.955598 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=462230f1837d49efa951a16d18453422&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6751788 May 24 10:07:18.956679 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59688) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:18.957423 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fc5dab8a-c64a-42c3-ac5c-3b262594d774,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:18.971892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:19.026253 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:19.042017 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c24d6eb5-5f72-44b2-8dcc-43fbd8be242b req-421781a6-c12c-46ba-9961-d9e7631a2cff tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1307299 May 24 10:07:19.069344 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:19.088584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-c29b2f1e-69e9-4abf-bb7b-bb45810e0f3e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:19.092928 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-c29b2f1e-69e9-4abf-bb7b-bb45810e0f3e service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.2025776 May 24 10:07:19.176779 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-8eadb74c-95c1-416e-9989-c7083bdc64a1 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 217 time: 0.0770612 May 24 10:07:19.207077 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-292a76e4-9694-4bcf-827d-75b38e656e5d req-994e0dee-cae3-4cef-b03a-1c65d6230a73 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:19.209798 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-292a76e4-9694-4bcf-827d-75b38e656e5d req-994e0dee-cae3-4cef-b03a-1c65d6230a73 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.1318748 May 24 10:07:19.280954 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-292a76e4-9694-4bcf-827d-75b38e656e5d req-79cc7da0-0edb-4e04-9a8f-9e65e6d3edb1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0619526 May 24 10:07:19.283341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-e9a1d8d0-8a80-467c-9d52-970efdfabcb3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:19.284867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:19.285271 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-e9a1d8d0-8a80-467c-9d52-970efdfabcb3 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.1012387 May 24 10:07:19.289150 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-df07f65b-3f86-43a3-a113-cefd43452d8d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:19.309730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fc5dab8a-c64a-42c3-ac5c-3b262594d774,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:19.318059 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:19.318852 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:19.320320 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/fc5dab8a-c64a-42c3-ac5c-3b262594d774 HTTP/1.1" status: 204 len: 168 time: 0.3600035 May 24 10:07:19.440872 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-e5aa6240-d8ae-49e9-a5d7-485bc4448c20 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:19.442972 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-e5aa6240-d8ae-49e9-a5d7-485bc4448c20 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1500471 May 24 10:07:19.624368 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-bbff3b58-e438-44c4-b17b-f508e77fa452 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1732519 May 24 10:07:19.984247 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f9effe85-ea66-4c07-8647-ea07910f2cd3 req-02d4f4a2-8fb8-48f5-a9fd-413af99173fd service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3531177 May 24 10:07:20.177271 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:20.284139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8dd25a81-1143-4d2f-a102-d30c4f1a5cd3 req-8f05eefd-5c14-4c89-96d5-fb70aa743870 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:20.287050 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8dd25a81-1143-4d2f-a102-d30c4f1a5cd3 req-8f05eefd-5c14-4c89-96d5-fb70aa743870 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1115313 May 24 10:07:20.344928 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8dd25a81-1143-4d2f-a102-d30c4f1a5cd3 req-fd9a84ee-7442-4e0f-9f28-0622b60c6303 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0515134 May 24 10:07:20.569994 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3054c627-17f0-4e41-8b83-2f834489df31 req-d157196c-bed1-4a2d-9842-e5f585377832 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:20.571448 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3054c627-17f0-4e41-8b83-2f834489df31 req-d157196c-bed1-4a2d-9842-e5f585377832 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.0911355 May 24 10:07:20.624387 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3054c627-17f0-4e41-8b83-2f834489df31 req-74f85160-e094-48b1-9aa7-e26bd746998f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0445704 May 24 10:07:21.552034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99c10752-1cbe-442e-a252-c773ac7506df req-e10f710d-cbd4-48ff-9178-dc2f08e93825 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:21.553684 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-99c10752-1cbe-442e-a252-c773ac7506df req-e10f710d-cbd4-48ff-9178-dc2f08e93825 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0918479 May 24 10:07:21.607846 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-99c10752-1cbe-442e-a252-c773ac7506df req-3671dc34-e1a9-4924-82b0-6377ec064269 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0440466 May 24 10:07:21.847255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1ccd338-c35e-44b0-b26d-692b48345382 req-cc2f8537-02ce-4d55-9386-160d9526987e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:21.849040 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d1ccd338-c35e-44b0-b26d-692b48345382 req-cc2f8537-02ce-4d55-9386-160d9526987e tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.0905991 May 24 10:07:21.899412 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d1ccd338-c35e-44b0-b26d-692b48345382 req-7ebace7a-f2dd-4633-a1a1-24c119484392 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0438981 May 24 10:07:22.808093 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01b66f71-62f9-461d-a5e9-27deec257031 req-deeb1a1d-e71c-4f66-815f-334c46fb9d00 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:22.810307 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b66f71-62f9-461d-a5e9-27deec257031 req-deeb1a1d-e71c-4f66-815f-334c46fb9d00 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0918198 May 24 10:07:22.860908 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b66f71-62f9-461d-a5e9-27deec257031 req-ca268d14-ef22-4837-a452-5c13d658c573 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0437896 May 24 10:07:24.107930 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68a3416d-d871-4e85-850f-11368d49a0c1 req-4d6c21e6-6dd4-4959-9463-a03e2f0c90d1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:24.109559 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-68a3416d-d871-4e85-850f-11368d49a0c1 req-4d6c21e6-6dd4-4959-9463-a03e2f0c90d1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1182117 May 24 10:07:24.162921 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-68a3416d-d871-4e85-850f-11368d49a0c1 req-a39de34d-5fe5-4c0a-b2d1-9a98a9b8940a tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0470130 May 24 10:07:25.489031 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b9590e6b-e792-4046-b09e-38e4d8e6beb3 req-134dbcd6-a493-4b75-a917-c916cc582600 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:25.491000 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b9590e6b-e792-4046-b09e-38e4d8e6beb3 req-134dbcd6-a493-4b75-a917-c916cc582600 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1397824 May 24 10:07:25.549560 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b9590e6b-e792-4046-b09e-38e4d8e6beb3 req-dacefed0-8121-4b22-ba8f-f5e32e6a2885 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0518672 May 24 10:07:25.885873 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:07:26.054147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:26.070617 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'mac_address': 'fa:16:3e:03:bf:e8', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 81, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:07:26.135328 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:26.155341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b5308ea7-0517-43e4-830b-7fd131b3a9c7 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:07:26.173025 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:26.239995 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:26.242628 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:26.330893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:26.334385 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:26.508466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['b5308ea7-0517-43e4-830b-7fd131b3a9c7'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:07:26.784121 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e8c25f6-8b86-4692-951d-dfe3b672c10a req-9deb452f-47e1-4fb5-a87c-29e5008faa39 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:26.785737 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5e8c25f6-8b86-4692-951d-dfe3b672c10a req-9deb452f-47e1-4fb5-a87c-29e5008faa39 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 944 time: 0.0985959 May 24 10:07:26.837280 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-5e8c25f6-8b86-4692-951d-dfe3b672c10a req-b6d80271-5696-4fe6-ba1a-82d503f15b65 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0450540 May 24 10:07:27.895184 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:07:28.115039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f59a00ec-ff9c-44b4-b952-25f2be9d20d0 req-dd6d4b3d-9715-4b8c-97c1-b52db7f24af4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:28.118009 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f59a00ec-ff9c-44b4-b952-25f2be9d20d0 req-dd6d4b3d-9715-4b8c-97c1-b52db7f24af4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 944 time: 0.1574986 May 24 10:07:28.119862 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port b5308ea7-0517-43e4-830b-7fd131b3a9c7 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:07:28.129374 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port b5308ea7-0517-43e4-830b-7fd131b3a9c7 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:07:28.130090 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:28.204920 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f59a00ec-ff9c-44b4-b952-25f2be9d20d0 req-f47cc87f-a234-412d-abee-2b4eb703bb22 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0796108 May 24 10:07:28.306051 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:28.338397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:28.350934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:28.407531 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'code': 200} May 24 10:07:28.428164 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:28.430322 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:28.503498 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:28.507274 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:28.897151 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85ba8d77-55c0-4b8d-95d1-9eb975a163e0 req-6a3b6710-5bf6-4b74-858c-bbb9c424ee61 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:28.898723 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-85ba8d77-55c0-4b8d-95d1-9eb975a163e0 req-6a3b6710-5bf6-4b74-858c-bbb9c424ee61 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.0830383 May 24 10:07:28.944954 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-85ba8d77-55c0-4b8d-95d1-9eb975a163e0 req-d3a7f769-92cb-409b-9481-52fd0523c149 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380712 May 24 10:07:29.435081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f421e7b6-d081-4fcf-9f3b-b72a586d3392 req-c834ae7f-6baf-4ef6-b47c-c8400bd93afa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:29.437459 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f421e7b6-d081-4fcf-9f3b-b72a586d3392 req-c834ae7f-6baf-4ef6-b47c-c8400bd93afa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0966084 May 24 10:07:29.487999 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f421e7b6-d081-4fcf-9f3b-b72a586d3392 req-5d93bd60-245c-47da-ad7e-3fbae05a2e5c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0442319 May 24 10:07:30.685975 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72259faf-418a-4c92-8070-60dbd4d0fd70 req-76593691-d164-45fb-9325-e9e127f72d88 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:30.688205 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72259faf-418a-4c92-8070-60dbd4d0fd70 req-76593691-d164-45fb-9325-e9e127f72d88 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0870862 May 24 10:07:30.738488 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-72259faf-418a-4c92-8070-60dbd4d0fd70 req-4ba75501-8262-44da-a80c-e8ceec69d69b tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0439689 May 24 10:07:30.956481 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea5229ad-9168-463b-8462-e8980b0cc0bd req-1455770f-2a81-470d-ba85-8726db0b20c9 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:30.958090 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ea5229ad-9168-463b-8462-e8980b0cc0bd req-1455770f-2a81-470d-ba85-8726db0b20c9 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0892608 May 24 10:07:31.009665 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ea5229ad-9168-463b-8462-e8980b0cc0bd req-7ea3c458-3eb4-492d-96db-b0471e1dba6f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0455465 May 24 10:07:31.050839 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59784) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:31.395559 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-dd409ded-ff28-47ca-9d00-859a343a5358 tempest-TestShelveInstance-1520094407 tempest-TestShelveInstance-1520094407-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:31.397189 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-dd409ded-ff28-47ca-9d00-859a343a5358 tempest-TestShelveInstance-1520094407 tempest-TestShelveInstance-1520094407-project-admin] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1093 time: 0.3449767 May 24 10:07:31.400134 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59796) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:31.405632 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'fixed_ip_address': '172.20.0.9'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:07:31.412100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:31.416966 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Usage tracker for resource:floatingip and tenant:ce96e2adbf10489aae7640c84f398b29 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:07:31.420635 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:07:31.427994 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:31.489021 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:07:31.542131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for floatingip, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:31.771552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:32.036135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Allocated IP 172.27.0.95 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/12af85ea-363f-46d1-a661-0c5c82780e12) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:07:32.269768 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36cf8b59-486a-4166-b4b2-b8aa50de24fe req-79929139-ef4b-460a-9555-f1c30cf38b0f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:32.271382 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-36cf8b59-486a-4166-b4b2-b8aa50de24fe req-79929139-ef4b-460a-9555-f1c30cf38b0f tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.2211215 May 24 10:07:32.280915 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:32.340614 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-36cf8b59-486a-4166-b4b2-b8aa50de24fe req-1ab1edd5-a5f5-44cb-b8b8-ee3682c3c5dc tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0552502 May 24 10:07:32.417193 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:32.422243 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:32.494530 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:07:32.498986 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:07:32.503856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:07:32.544196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'Port': ['ID=12af85ea-363f-46d1-a661-0c5c82780e12,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:32.564128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.564595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.564975 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.565365 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.565726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.566089 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.566441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.566794 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.567148 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.567499 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.567854 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.568196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:32.573906 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:33.196124 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:33.302774 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c901e762-084a-4758-9bab-d2908aff8090 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:33.339191 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:33.593774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:33.641908 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7be8ef2c-0964-40fd-8b42-fbe663b60159 req-ea0e01f1-41f4-4d39-a37c-3798161f782a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:33.643489 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7be8ef2c-0964-40fd-8b42-fbe663b60159 req-ea0e01f1-41f4-4d39-a37c-3798161f782a tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.1728282 May 24 10:07:33.650678 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:33.654700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:33.693814 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7be8ef2c-0964-40fd-8b42-fbe663b60159 req-eb340fc5-cf1a-46fd-8d81-f81627fc1b51 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0444033 May 24 10:07:33.779988 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event updated for resources: {'Port': ['ID=12af85ea-363f-46d1-a661-0c5c82780e12,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:33.784810 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:33.796237 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:34.050379 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Floating IP 4db86f31-88c4-47f1-a287-fac820ccc2bf associated. External IP: 172.27.0.95, port: b5308ea7-0517-43e4-830b-7fd131b3a9c7. May 24 10:07:34.147555 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-79f4db08-f128-4030-b88d-a09f09a9f8c5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:34.149179 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-79f4db08-f128-4030-b88d-a09f09a9f8c5 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.2798443 May 24 10:07:34.547684 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Deleting port 08d1ba31-f09e-4772-a2c3-16a65ba1761c {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:34.548499 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:34.666554 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Router 91d55ed4-1190-40a0-83e1-ece622fa9c2e has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:34.932779 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] For port 08d1ba31-f09e-4772-a2c3-16a65ba1761c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=08d1ba31-f09e-4772-a2c3-16a65ba1761c,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:34.933167 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:35.280304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:35.292069 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Sync routers for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:35.306787 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:35.307436 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:35.415492 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b95e68d-01be-41eb-bfdd-15351e5a1682 req-54217769-3b8e-42d0-ae95-ca998a332a40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:35.417214 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8b95e68d-01be-41eb-bfdd-15351e5a1682 req-54217769-3b8e-42d0-ae95-ca998a332a40 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 945 time: 0.4926834 May 24 10:07:35.491144 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:35.491507 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:35.491828 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Floating IP 9480b242-ba4a-420b-bc39-da7c46eed2f8 disassociated. External IP: 172.27.0.96, port: 08d1ba31-f09e-4772-a2c3-16a65ba1761c. May 24 10:07:35.492175 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Calling delete_port for 08d1ba31-f09e-4772-a2c3-16a65ba1761c owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:35.515090 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8b95e68d-01be-41eb-bfdd-15351e5a1682 req-c3461d58-a601-4e37-9f3a-837ae8a0db15 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups?id=fcf796c6-f87f-462a-9699-d361158092a8&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0792999 May 24 10:07:35.528659 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:35.534939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:35.639404 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 4.2380013 May 24 10:07:35.642952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:35.744447 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-13ee62cf-3208-4c7e-8c3f-d72b478fc8b9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-205366445&fields=id&tenant_id=4b47c7cc67c545eab92ea86cde1abac7 HTTP/1.1" status: 200 len: 267 time: 0.0496516 May 24 10:07:35.809198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'status': 'completed', 'code': 200} May 24 10:07:35.947415 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-6914efbb-c5ab-40ac-a861-dd1635745666 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:35.949879 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-6914efbb-c5ab-40ac-a861-dd1635745666 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.1062663 May 24 10:07:35.960005 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-a37f18d5-84a2-44c0-896e-3014cebc1ebb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:35.966673 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:35.969398 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-a37f18d5-84a2-44c0-896e-3014cebc1ebb tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 984 time: 0.2120194 May 24 10:07:35.986925 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8d3c1932-cba9-4d35-ab6b-89bb5ea947c1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:35.998216 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Request body: {'port': {'security_groups': ['e83316d5-9d97-4e86-bce6-f8bde9c85f77']}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:07:36.051973 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:36.065346 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a8582d0f-3766-459e-aabe-48c72788cb14 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:36.093402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'Port': ['ID=08d1ba31-f09e-4772-a2c3-16a65ba1761c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:36.098997 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:36.239504 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '4db86f31-88c4-47f1-a287-fac820ccc2bf', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'floating_ip_address': '172.27.0.95', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 532, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'mac_address': 'fa:16:3e:03:bf:e8', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:07:33Z', 'updated_at': '2021-05-24T10:07:33Z', 'revision_number': 0, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:36.245294 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:36.247648 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Checking router: 91d55ed4-1190-40a0-83e1-ece622fa9c2e for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:36.260378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:36.261038 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:36.340465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-732a9fe5-e219-4c76-871d-3f66415c6bc4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:36.341986 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-732a9fe5-e219-4c76-871d-3f66415c6bc4 service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.3693764 May 24 10:07:36.346923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:36.505516 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-6a379949-bc64-44ee-879a-39bd62b83224 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 998 time: 0.1535058 May 24 10:07:36.607132 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:36.646268 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:36.661571 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:36.695131 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:36.698523 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:36.750667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] No DVR routers for this DVR port 08d1ba31-f09e-4772-a2c3-16a65ba1761c on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:07:36.759091 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:36.786729 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-c36e1a60-3084-4b13-80f6-4081a0c243ef service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:36.788923 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-c36e1a60-3084-4b13-80f6-4081a0c243ef service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.2716084 May 24 10:07:36.830484 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-91581dc1-95b0-4e7e-a225-857a6ddb086e req-1341571e-37e1-4f87-a8e3-bdf10e40984c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Synchronizing usage tracker for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:07:36.877392 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-288e7b8b-6410-457b-b543-b6bb9354b023 req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.95 "DELETE /v2.0/ports/08d1ba31-f09e-4772-a2c3-16a65ba1761c HTTP/1.1" status: 204 len: 173 time: 2.7194963 May 24 10:07:36.949819 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:36.968298 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:36.982574 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-91581dc1-95b0-4e7e-a225-857a6ddb086e req-1341571e-37e1-4f87-a8e3-bdf10e40984c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Unset dirty status for tenant:3f91ad6930924a8d81ee21ec1c490828 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:07:36.986092 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-91581dc1-95b0-4e7e-a225-857a6ddb086e req-1341571e-37e1-4f87-a8e3-bdf10e40984c tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 210 time: 0.2007647 May 24 10:07:37.007793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.022527 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-4002363d-623d-4683-86ef-884802089ecc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:37.024175 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-4002363d-623d-4683-86ef-884802089ecc service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2256777 May 24 10:07:37.125150 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.134320 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2a7d00be-9ce9-4780-b250-2aebc870c445 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:37.171587 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['fb9be3d2-4022-41e3-825a-ddf6079de0ba'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:07:37.233713 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:37.234385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.236322 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b6a1847b-9219-45f8-bc2c-0f8b587a6974 req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "PUT /v2.0/ports/fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 942 time: 1.2652292 May 24 10:07:37.239237 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:37.243757 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59822) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:37.276660 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-cf52c5ba-787f-46a5-b21f-f7445121c6a1 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2451119 May 24 10:07:37.278135 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event updated for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=7']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:37.281018 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.282269 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:37.385806 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['fb9be3d2-4022-41e3-825a-ddf6079de0ba'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:07:37.410617 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:37.416254 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:37.428014 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:37.428960 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:37.437846 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:07:37.511202 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f480d267-f7a8-46d5-be1a-40b3da3460de req-6bf886c6-3da1-4d49-b561-70965747b36d service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2273312 May 24 10:07:37.516276 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '4db86f31-88c4-47f1-a287-fac820ccc2bf', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'floating_ip_address': '172.27.0.95', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 532, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'mac_address': 'fa:16:3e:03:bf:e8', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:07:33Z', 'updated_at': '2021-05-24T10:07:33Z', 'revision_number': 0, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:37.554227 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-76a9a6d8-42d0-4783-8289-8abc307eccc2 None None] The sync data for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi is: [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '4db86f31-88c4-47f1-a287-fac820ccc2bf', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'floating_ip_address': '172.27.0.95', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 532, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'mac_address': 'fa:16:3e:03:bf:e8', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:07:33Z', 'updated_at': '2021-05-24T10:07:33Z', 'revision_number': 0, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:37.629864 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.680305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.697822 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:37.751814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-1bfbf8c3-3784-4a35-87e7-aaacde5f76f0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:37.754788 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-1bfbf8c3-3784-4a35-87e7-aaacde5f76f0 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1098 time: 0.3889053 May 24 10:07:37.810796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'name': 'network-vif-deleted', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:37.825261 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-a2b6e886-c825-4983-854d-5508f79f2a60 None None] Provisioning complete for port fb9be3d2-4022-41e3-825a-ddf6079de0ba triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:07:37.825841 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a2b6e886-c825-4983-854d-5508f79f2a60 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:37.931059 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 08d1ba31-f09e-4772-a2c3-16a65ba1761c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:37.973497 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '91e8cab0-6add-4148-873b-9f1564abc530', 'name': 'network-vif-deleted', 'tag': '08d1ba31-f09e-4772-a2c3-16a65ba1761c', 'status': 'completed', 'code': 200} May 24 10:07:38.000013 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d23dd58-2fa6-4a3d-bfda-08bd3aa80002 None None] New status for floating IP 4db86f31-88c4-47f1-a287-fac820ccc2bf: ACTIVE {{(pid=109114) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:07:38.029273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-f0fbd2e6-13cf-43ff-ae81-aa18657edb68 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:38.033087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-f0fbd2e6-13cf-43ff-ae81-aa18657edb68 service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.2707202 May 24 10:07:38.053325 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 08d1ba31-f09e-4772-a2c3-16a65ba1761c {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:38.053827 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 08d1ba31-f09e-4772-a2c3-16a65ba1761c not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:38.105829 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:38.128230 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1bf21ae3-0c2b-4f8f-b6b3-c346b5468c83 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:38.175795 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=74db5aed-164a-4d86-968e-bc83de180d04,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:38.188543 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:38.244137 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-2b295551-7f57-400f-9106-75b565e9cff8 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 1000 time: 0.2037799 May 24 10:07:38.245657 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:38.249876 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "DELETE /v2.0/security-groups/74db5aed-164a-4d86-968e-bc83de180d04 HTTP/1.1" status: 204 len: 168 time: 1.0026398 May 24 10:07:38.250788 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59832) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:38.348949 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-46fc637a-1b1e-4eef-b9e2-685db5d3b823 req-61a4cc18-765e-4d00-94ca-ae4f908b2041 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/ports?device_id=91e8cab0-6add-4148-873b-9f1564abc530 HTTP/1.1" status: 200 len: 210 time: 0.0939667 May 24 10:07:38.393257 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-155f6765-2cf4-4f08-9c16-003837ebfd03 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:38.396304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-155f6765-2cf4-4f08-9c16-003837ebfd03 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.1423526 May 24 10:07:38.429454 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:38.434991 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3b75f0a0-72b7-4cf1-82d9-4b64e3c79d13 None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:38Z', 'revision_number': 4, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'ed75d389-b3db-45f5-bf82-87fe5209a3e9', 'name': '', 'network_id': 'e28486c2-462f-406b-8c64-3246ae76f56c', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'mac_address': 'fa:16:3e:86:08:97', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_interface', 'standard_attr_id': 92, 'fixed_ips': [{'subnet_id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:36Z', 'updated_at': '2021-05-24T10:00:51Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'subnets': [{'id': '59e96feb-f5d8-4b1e-b08d-b93c34f5461c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:38.509021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-29a9d23e-7216-44c6-88aa-a4e787aef152 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:38.510745 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-29a9d23e-7216-44c6-88aa-a4e787aef152 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1077619 May 24 10:07:38.677363 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-2299dd9e-99c5-47b1-98a4-092c9069f4bf service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1564155 May 24 10:07:38.737282 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Deleting port f0a1e941-ba29-43f9-8b64-db23166bf9ed {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:38.737991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:38.776722 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d401c8be-3014-4967-acc3-0b756418c454 None None] New status for floating IP 9480b242-ba4a-420b-bc39-da7c46eed2f8: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:07:38.822880 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:38.877304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-34abb223-025c-4cd2-8f9f-d0a83e110dfe req-c9a20e39-63db-4257-bb63-50da1859e8bc service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1919298 May 24 10:07:38.906864 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:38.907298 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.043938 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Calling delete_port for f0a1e941-ba29-43f9-8b64-db23166bf9ed owned by network:floatingip {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:39.256694 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.263191 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:39.311551 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event deleted for resources: {'Port': ['ID=f0a1e941-ba29-43f9-8b64-db23166bf9ed,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:39.347828 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:39.426074 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks [] for floatingip, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.426653 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Floating IP 446b2090-daad-4112-afca-bdc8632398c4 disassociated (deleted). External IP: 172.27.0.64, port: fb9be3d2-4022-41e3-825a-ddf6079de0ba. May 24 10:07:39.678279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59840) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:39.787106 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.826471 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.922465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Router 7ead47d0-d22c-480f-8edb-a291a66ba849 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:39.982568 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:39.988588 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c5896fbd-8be4-41a7-a349-54bf00baf197 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:40.005573 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=fcf796c6-f87f-462a-9699-d361158092a8,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:40.007987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:40.107092 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:40.108285 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "DELETE /v2.0/security-groups/fcf796c6-f87f-462a-9699-d361158092a8 HTTP/1.1" status: 204 len: 168 time: 0.4285314 May 24 10:07:40.111071 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59844) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:40.128574 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.api.v2.resource [None req-c0ded16c-026f-42cc-b90f-a7ad9b53f9e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] show failed (client error): The resource could not be found. May 24 10:07:40.129663 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c0ded16c-026f-42cc-b90f-a7ad9b53f9e4 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "GET /v2.0/security-groups/fcf796c6-f87f-462a-9699-d361158092a8 HTTP/1.1" status: 404 len: 347 time: 0.0175583 May 24 10:07:40.132874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59848) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:40.136340 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:40.149000 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Sync routers for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:40.160877 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:40.161637 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:40.220325 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:40.309209 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "DELETE /v2.0/floatingips/446b2090-daad-4112-afca-bdc8632398c4 HTTP/1.1" status: 204 len: 168 time: 2.0560145 May 24 10:07:40.309910 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:40.372651 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:07:40.404921 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:40.698128 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01b872cb-8c34-493c-9e15-6de9759557ba req-15ca4d7b-1da9-4d5a-a8a5-2a635e8f43ea service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:40.700493 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-15ca4d7b-1da9-4d5a-a8a5-2a635e8f43ea service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 1098 time: 0.2715683 May 24 10:07:40.760134 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Deleting port d23a3389-d1d4-4243-951e-265997c76053 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:40.760472 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:40.864982 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:40.870180 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Checking router: 7ead47d0-d22c-480f-8edb-a291a66ba849 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:40.887634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:40.887634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:40.957791 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01b872cb-8c34-493c-9e15-6de9759557ba req-a2866c19-e7fd-4dc1-9e98-78fbed51cb1b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:40.959291 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-a2866c19-e7fd-4dc1-9e98-78fbed51cb1b service neutron] 10.222.0.95 "GET /v2.0/networks?id=d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 200 len: 915 time: 0.2517855 May 24 10:07:41.112238 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:41.242087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-193f8046-804a-4474-abda-8797a3a80a9c service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 200 len: 217 time: 0.2691188 May 24 10:07:41.244757 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:41.245113 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:41.427524 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Calling delete_port for d23a3389-d1d4-4243-951e-265997c76053 owned by network:floatingip {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:41.447927 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01b872cb-8c34-493c-9e15-6de9759557ba req-3a081e13-26e8-46cc-811d-6a84fc9b4216 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:41.449955 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-3a081e13-26e8-46cc-811d-6a84fc9b4216 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 200 len: 854 time: 0.1890945 May 24 10:07:41.560021 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01b872cb-8c34-493c-9e15-6de9759557ba req-6cf22778-e698-4f5f-ac3f-f4d6b3391aa7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:41.561568 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-6cf22778-e698-4f5f-ac3f-f4d6b3391aa7 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=d34a1523-ea50-45ad-a897-459ffff7af90&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1045642 May 24 10:07:41.631837 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:41.636159 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae9d7759-cd7c-4cce-8ca1-f24472602625 None None] The sync data for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi is: [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:04:55Z', 'revision_number': 4, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'eca6df41-ad3d-4f1a-9a5c-cd20b4f97938', 'name': '', 'network_id': 'd34a1523-ea50-45ad-a897-459ffff7af90', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'mac_address': 'fa:16:3e:91:06:4e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_interface', 'standard_attr_id': 426, 'fixed_ips': [{'subnet_id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:54Z', 'updated_at': '2021-05-24T10:05:02Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'subnets': [{'id': 'a1ecfc88-eaf2-426e-bb90-f6a474fb5991', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:41.759307 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-5650e294-5ee9-4fea-8528-2baaa69de46c service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1624534 May 24 10:07:41.827884 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:41.839985 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:41.923391 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a4476258-4f0e-4911-9eef-23b58fe4014e None None] New status for floating IP 446b2090-daad-4112-afca-bdc8632398c4: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:07:41.970617 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Pushing event deleted for resources: {'Port': ['ID=d23a3389-d1d4-4243-951e-265997c76053,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:41.973038 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:42.117370 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-01b872cb-8c34-493c-9e15-6de9759557ba req-5b86acdf-76f2-490b-ad1e-eb78b3f91284 service neutron] 10.222.0.95 "GET /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3495324 May 24 10:07:42.127083 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks [] for floatingip, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:42.154836 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:42.156275 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] 10.222.0.67 "DELETE /v2.0/floatingips/9480b242-ba4a-420b-bc39-da7c46eed2f8 HTTP/1.1" status: 204 len: 168 time: 2.0217915 May 24 10:07:45.868775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bfaca16-ae5a-4f20-9495-785469031a8c req-7b0499d5-72ea-4802-8451-fd1877d5e54e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:45.869702 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8bfaca16-ae5a-4f20-9495-785469031a8c req-7b0499d5-72ea-4802-8451-fd1877d5e54e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:07:45.947772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-8bfaca16-ae5a-4f20-9495-785469031a8c req-7b0499d5-72ea-4802-8451-fd1877d5e54e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:07:45.949119 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8bfaca16-ae5a-4f20-9495-785469031a8c req-7b0499d5-72ea-4802-8451-fd1877d5e54e tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1818168 May 24 10:07:45.998228 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8bfaca16-ae5a-4f20-9495-785469031a8c req-9d8e54c0-6150-40fe-a1ca-36238229dbc9 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0426660 May 24 10:07:47.021886 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59910) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:47.200171 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c6b28cf-52b2-4fb1-b8d9-55673305e4bd req-f9eb66aa-3a54-48a9-b4fc-b80bd4c21552 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:47.201783 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5c6b28cf-52b2-4fb1-b8d9-55673305e4bd req-f9eb66aa-3a54-48a9-b4fc-b80bd4c21552 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.0910199 May 24 10:07:47.341411 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-5c6b28cf-52b2-4fb1-b8d9-55673305e4bd req-36f9359a-8b50-4637-9c3b-aa667c0747d6 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Synchronizing usage tracker for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:07:47.524691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-5c6b28cf-52b2-4fb1-b8d9-55673305e4bd req-36f9359a-8b50-4637-9c3b-aa667c0747d6 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Unset dirty status for tenant:4b47c7cc67c545eab92ea86cde1abac7 on resource:security_group {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:07:47.525930 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5c6b28cf-52b2-4fb1-b8d9-55673305e4bd req-36f9359a-8b50-4637-9c3b-aa667c0747d6 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3170116 May 24 10:07:47.909600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-e62c5665-caec-48f4-b376-1c1c84378b6f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:47.911047 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-e62c5665-caec-48f4-b376-1c1c84378b6f tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.95 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.3724191 May 24 10:07:48.144393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Deleting port fb9be3d2-4022-41e3-825a-ddf6079de0ba {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:48.145314 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:48.303222 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:48.388994 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] For port fb9be3d2-4022-41e3-825a-ddf6079de0ba, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=fb9be3d2-4022-41e3-825a-ddf6079de0ba,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:48.389395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:48.533030 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Calling delete_port for fb9be3d2-4022-41e3-825a-ddf6079de0ba owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:48.797395 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea96d5c9-5d9f-42af-b942-5b78b4d6215f req-b078e726-de10-4054-9671-52f64c7565e4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:48.799999 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ea96d5c9-5d9f-42af-b942-5b78b4d6215f req-b078e726-de10-4054-9671-52f64c7565e4 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 945 time: 0.1251519 May 24 10:07:48.881964 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ea96d5c9-5d9f-42af-b942-5b78b4d6215f req-38d2d332-6c3f-42d8-b080-0e4eb516d0d8 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/security-groups?id=e83316d5-9d97-4e86-bce6-f8bde9c85f77&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0749173 May 24 10:07:48.998917 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:49.010279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ba9f880-8756-48e5-bccc-40b1eecce785 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:49.069312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Pushing event deleted for resources: {'Port': ['ID=fb9be3d2-4022-41e3-825a-ddf6079de0ba,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:49.072378 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:49.160979 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] No DVR routers for this DVR port fb9be3d2-4022-41e3-825a-ddf6079de0ba on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:07:49.165835 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:49.169371 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'name': 'network-vif-deleted', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:49.189522 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Deleting port ed75d389-b3db-45f5-bf82-87fe5209a3e9 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:49.189947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:49.282676 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '354f2aa8-dfb0-4ea5-8240-67197c5b4362', 'name': 'network-vif-deleted', 'tag': 'fb9be3d2-4022-41e3-825a-ddf6079de0ba', 'status': 'completed', 'code': 200} May 24 10:07:49.287299 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5890cfb6-ec92-4dc0-9a72-89d52df0b601 req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.95 "DELETE /v2.0/ports/fb9be3d2-4022-41e3-825a-ddf6079de0ba HTTP/1.1" status: 204 len: 173 time: 1.3700240 May 24 10:07:49.340213 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] For port ed75d389-b3db-45f5-bf82-87fe5209a3e9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ed75d389-b3db-45f5-bf82-87fe5209a3e9,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:49.340512 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:49.600495 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Calling delete_port for ed75d389-b3db-45f5-bf82-87fe5209a3e9 owned by network:router_interface {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:49.931302 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device fb9be3d2-4022-41e3-825a-ddf6079de0ba no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:50.000352 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: fb9be3d2-4022-41e3-825a-ddf6079de0ba {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:50.000955 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device fb9be3d2-4022-41e3-825a-ddf6079de0ba not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:50.029105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:50.037920 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-985e0e85-ce85-4909-982c-66bd93a55b2d tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:50.128019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Pushing event deleted for resources: {'Port': ['ID=ed75d389-b3db-45f5-bf82-87fe5209a3e9,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:50.170924 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:50.205112 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.167s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:50.238717 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f36b949f-5666-4308-8d5f-6feffd190b78 req-f210b8b0-ebf2-4365-9436-904d099b166c tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] 10.222.0.67 "GET /v2.0/ports?device_id=354f2aa8-dfb0-4ea5-8240-67197c5b4362 HTTP/1.1" status: 200 len: 210 time: 0.1641247 May 24 10:07:50.539095 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:50.962637 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Router 1ec02266-2164-4194-a981-4dcbb8771c01 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:51.285168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:51.300200 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Sync routers for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:51.303852 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] 10.222.0.67 "PUT /v2.0/routers/1ec02266-2164-4194-a981-4dcbb8771c01/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 4.2803798 May 24 10:07:51.308007 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59930) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:51.314462 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:51.315494 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:51.591494 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:51.657573 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks [] for router, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:51.894328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:52.031028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Deleting port cd9a22d0-5e50-4294-9642-050e775c1046 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:52.031370 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:52.051077 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:07:49Z', 'revision_number': 5, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port': {'id': 'cd9a22d0-5e50-4294-9642-050e775c1046', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b6:91:c3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'device_owner': 'network:router_gateway', 'standard_attr_id': 64, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:28Z', 'updated_at': '2021-05-24T10:00:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:52.055972 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Checking router: 1ec02266-2164-4194-a981-4dcbb8771c01 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:52.068673 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Routers {'1ec02266-2164-4194-a981-4dcbb8771c01'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:52.069389 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['1ec02266-2164-4194-a981-4dcbb8771c01'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:52.226428 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] For port cd9a22d0-5e50-4294-9642-050e775c1046, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=cd9a22d0-5e50-4294-9642-050e775c1046,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:52.228938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:52.325818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] Set router 1ec02266-2164-4194-a981-4dcbb8771c01 gateway port host: None {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:52.362997 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Calling delete_port for cd9a22d0-5e50-4294-9642-050e775c1046 owned by network:router_gateway {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:52.437290 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:07:51Z', 'revision_number': 6, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:52.442206 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7c04e8e2-8712-4a12-ac99-a7bcd68d4c1e None None] The sync data for ids ['1ec02266-2164-4194-a981-4dcbb8771c01'] in dv-cinder-791239-2-iscsi is: [{'id': '1ec02266-2164-4194-a981-4dcbb8771c01', 'name': 'tempest-AttachVolumeTestJSON-127484096-router', 'tenant_id': '3f91ad6930924a8d81ee21ec1c490828', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:07:51Z', 'revision_number': 6, 'project_id': '3f91ad6930924a8d81ee21ec1c490828', 'gw_port_host': None}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:52.628045 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:52.632341 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9dffa79a-52e7-45d9-9530-adef9630f4f2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:52.682265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Pushing event deleted for resources: {'Port': ['ID=cd9a22d0-5e50-4294-9642-050e775c1046,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:52.685680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:52.811265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:52.933710 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ed75d389-b3db-45f5-bf82-87fe5209a3e9 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:52.984277 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: ed75d389-b3db-45f5-bf82-87fe5209a3e9 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:52.984614 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ed75d389-b3db-45f5-bf82-87fe5209a3e9 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:53.060388 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.063092 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.171149 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.171762 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Fanout notify agent at l3_agent the message router_deleted on router 1ec02266-2164-4194-a981-4dcbb8771c01 {{(pid=109113) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:07:53.187726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.189608 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] 10.222.0.67 "DELETE /v2.0/routers/1ec02266-2164-4194-a981-4dcbb8771c01 HTTP/1.1" status: 204 len: 168 time: 1.8798919 May 24 10:07:53.191634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59976) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:53.276318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 59982) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:53.289811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Deleting subnet 59e96feb-f5d8-4b1e-b08d-b93c34f5461c {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:07:53.294047 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.383894 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:53.630227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.137035 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.210922 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.220620 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4f74bcaa-3dce-40ca-a7a1-fef6c4e27e25 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:54.294523 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Pushing event deleted for resources: {'Subnet': ['ID=59e96feb-f5d8-4b1e-b08d-b93c34f5461c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:54.304102 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.308627 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-abfcdfa2-3336-4f16-b900-9e5110410712 None None] Network e28486c2-462f-406b-8c64-3246ae76f56c requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:07:54.310874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:54.315714 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-f8ac755e-dd4e-4bb4-8bec-790a3443244a admin admin] 10.222.0.67 "DELETE /v2.0/subnets/59e96feb-f5d8-4b1e-b08d-b93c34f5461c HTTP/1.1" status: 204 len: 168 time: 1.1226165 May 24 10:07:54.318234 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 59992) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:54.463502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.534944 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30105698-3106-4221-83e0-6f4fb843552c req-470fb7d8-6893-4066-9b4e-b2fb90fa98cd tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:54.534944 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-30105698-3106-4221-83e0-6f4fb843552c req-470fb7d8-6893-4066-9b4e-b2fb90fa98cd tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.2052310 May 24 10:07:54.564402 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device cd9a22d0-5e50-4294-9642-050e775c1046 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:54.571690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Deleting port e0e331ae-89b8-435b-bceb-b7531603a358 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:54.578531 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.606890 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: cd9a22d0-5e50-4294-9642-050e775c1046 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:54.607220 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device cd9a22d0-5e50-4294-9642-050e775c1046 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:54.641638 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-30105698-3106-4221-83e0-6f4fb843552c req-6d3a6dad-4ed4-4cf7-8d53-fd67868cf70c tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0996215 May 24 10:07:54.742328 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] DHCP port deletion for e28486c2-462f-406b-8c64-3246ae76f56c request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:07:54.742565 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Deleting port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:54.742565 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.784172 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] For port e0e331ae-89b8-435b-bceb-b7531603a358, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e0e331ae-89b8-435b-bceb-b7531603a358,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:54.784759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.805409 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Deleting port e0e331ae-89b8-435b-bceb-b7531603a358 {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:54.805939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.913711 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] For port e0e331ae-89b8-435b-bceb-b7531603a358, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e0e331ae-89b8-435b-bceb-b7531603a358,segment=NetworkSegment(25064880-e56c-4a2f-b56c-21997837afc6),segment_id=25064880-e56c-4a2f-b56c-21997837afc6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:54.914602 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.962442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] For port eca6df41-ad3d-4f1a-9a5c-cd20b4f97938, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:54.962775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:54.975143 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Calling delete_port for e0e331ae-89b8-435b-bceb-b7531603a358 owned by network:dhcp {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:55.086740 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.093381 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:55.096092 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Calling delete_port for eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 owned by network:router_interface {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:55.118650 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Calling delete_port for e0e331ae-89b8-435b-bceb-b7531603a358 owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:55.192654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Pushing event deleted for resources: {'Port': ['ID=e0e331ae-89b8-435b-bceb-b7531603a358,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:55.197824 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] No DVR routers for this DVR port e0e331ae-89b8-435b-bceb-b7531603a358 on host dv-cinder-791239-2-iscsi {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:07:55.201600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:55.292046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.297009 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-4d183b17-4bf8-4ff3-9acb-fee0d826f0ef None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:55.345004 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Pushing event deleted for resources: {'Port': ['ID=e0e331ae-89b8-435b-bceb-b7531603a358,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:55.350967 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] No DVR routers for this DVR port e0e331ae-89b8-435b-bceb-b7531603a358 on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:07:55.357683 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.358147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.358556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.358889 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.359213 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.359539 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.359861 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.360179 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.360496 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.360811 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.361156 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.361469 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:07:55.365279 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:55.414844 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.434706 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['e28486c2-462f-406b-8c64-3246ae76f56c']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:55.448529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '25064880-e56c-4a2f-b56c-21997837afc6') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:55.449514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '25064880-e56c-4a2f-b56c-21997837afc6') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:55.462737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.486668 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.497208 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Releasing vlan 61 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:07:55.500667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:07:55.512724 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:07:55.701202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-2cbc7191-b955-4bd3-b3bb-9a0e00087e98 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:55.703854 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-2cbc7191-b955-4bd3-b3bb-9a0e00087e98 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.1197412 May 24 10:07:55.764991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.771821 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-161e3372-15d6-4817-a551-7ba6fba9c9d0 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:55.775736 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.814966 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0ab4a797-ebbf-4c05-b678-82dd82936db2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:55.824344 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Pushing event deleted for resources: {'Port': ['ID=eca6df41-ad3d-4f1a-9a5c-cd20b4f97938,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:55.839139 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:55.854411 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:55.887573 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Pushing event deleted for resources: {'Network': ['ID=e28486c2-462f-406b-8c64-3246ae76f56c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:55.909559 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:56.030057 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7afd92e2-b4cf-4aab-85a5-de2ba3df3187 req-0301848c-4095-4933-8338-3a2381a22e4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:07:56.031655 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7afd92e2-b4cf-4aab-85a5-de2ba3df3187 req-0301848c-4095-4933-8338-3a2381a22e4f tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 947 time: 0.2261655 May 24 10:07:56.034445 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Deleting port 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:56.034789 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.067858 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] 10.222.0.67 "DELETE /v2.0/networks/e28486c2-462f-406b-8c64-3246ae76f56c HTTP/1.1" status: 204 len: 168 time: 1.7479193 May 24 10:07:56.180987 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7afd92e2-b4cf-4aab-85a5-de2ba3df3187 req-0b882969-eeda-4330-aa6f-4885553d8ed6 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups?id=b0a9d06a-7b60-46b7-9ded-3120b0dc1680&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1310327 May 24 10:07:56.227206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] For port 00cb560b-b02f-42f4-9ee8-09089f5ae590, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=00cb560b-b02f-42f4-9ee8-09089f5ae590,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:56.230750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.249525 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.530539 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60002) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:56.695030 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b81ca8a1-6166-476a-bdd1-284a60288ffa admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=3f91ad6930924a8d81ee21ec1c490828&name=default HTTP/1.1" status: 200 len: 2975 time: 0.1661873 May 24 10:07:56.698268 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60006) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:56.706485 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.707258 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.707623 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Floating IP 9453c2e9-84b1-4b24-856e-dc0745e22ee7 disassociated. External IP: 172.27.0.60, port: 00cb560b-b02f-42f4-9ee8-09089f5ae590. May 24 10:07:56.708177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Calling delete_port for 00cb560b-b02f-42f4-9ee8-09089f5ae590 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:56.732865 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e0e331ae-89b8-435b-bceb-b7531603a358 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:56.786172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Router 7ead47d0-d22c-480f-8edb-a291a66ba849 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:56.797991 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: e0e331ae-89b8-435b-bceb-b7531603a358 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:56.799007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e0e331ae-89b8-435b-bceb-b7531603a358 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:56.908398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.947722 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:56.954902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-14b4d44c-3e8a-49d6-b4a7-cfbc7b09741c service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:57.009837 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'Port': ['ID=00cb560b-b02f-42f4-9ee8-09089f5ae590,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:57.011982 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.013185 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:57.058874 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:57.074008 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Sync routers for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:57.082957 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-84bef3a5-e2ec-474d-873b-b08a8f922a69 admin admin] 10.222.0.67 "PUT /v2.0/routers/7ead47d0-d22c-480f-8edb-a291a66ba849/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.8050172 May 24 10:07:57.085847 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60010) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:57.093492 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:57.094386 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:57.226584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:07:57.346359 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:57.479121 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:07:57.498185 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:07:57.519896 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:57.520836 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:57.521083 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.525245 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f4988368-2bd0-4df7-ac59-3591aca62a6e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:57.585394 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=623bc222-7568-4ac6-9179-b8d061289a25,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:57.589094 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-679a4c5a-a4e8-4a47-b75f-873f5028b3eb req-08b00dbf-434b-4d33-b267-de7ceeef233b tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.0997081 May 24 10:07:57.592672 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:57.596490 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.597585 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/623bc222-7568-4ac6-9179-b8d061289a25 HTTP/1.1" status: 204 len: 168 time: 0.8977773 May 24 10:07:57.613479 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks [] for router, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.668934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] No DVR routers for this DVR port 00cb560b-b02f-42f4-9ee8-09089f5ae590 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:07:57.673042 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.676265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-deleted', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:07:57.741833 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '812a56d8-d836-470a-b693-994765367aec', 'name': 'network-vif-deleted', 'tag': '00cb560b-b02f-42f4-9ee8-09089f5ae590', 'status': 'completed', 'code': 200} May 24 10:07:57.754347 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3f25924a-a3de-4b87-854e-61a18235aa6d req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.95 "DELETE /v2.0/ports/00cb560b-b02f-42f4-9ee8-09089f5ae590 HTTP/1.1" status: 204 len: 173 time: 2.0383728 May 24 10:07:57.867254 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:57.925219 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:07:55Z', 'revision_number': 5, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port': {'id': '8c72ce53-67f4-42a6-beef-bb11cea7beb2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ce:c1:46', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'device_owner': 'network:router_gateway', 'standard_attr_id': 425, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:04:51Z', 'updated_at': '2021-05-24T10:04:56Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:57.930078 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Checking router: 7ead47d0-d22c-480f-8edb-a291a66ba849 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:57.940845 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Routers {'7ead47d0-d22c-480f-8edb-a291a66ba849'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:57.941542 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['7ead47d0-d22c-480f-8edb-a291a66ba849'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:57.985635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:58.051973 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 00cb560b-b02f-42f4-9ee8-09089f5ae590 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:58.052489 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 00cb560b-b02f-42f4-9ee8-09089f5ae590 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:58.060347 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:58.069639 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Deleting port 8c72ce53-67f4-42a6-beef-bb11cea7beb2 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:07:58.070022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:58.162610 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] Set router 7ead47d0-d22c-480f-8edb-a291a66ba849 gateway port host: None {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:58.245230 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] For port 8c72ce53-67f4-42a6-beef-bb11cea7beb2, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8c72ce53-67f4-42a6-beef-bb11cea7beb2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:07:58.246140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:58.418293 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Calling delete_port for 8c72ce53-67f4-42a6-beef-bb11cea7beb2 owned by network:router_gateway {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:07:58.446096 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:07:57Z', 'revision_number': 6, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:58.453127 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0521b736-44c8-45f4-a79c-28a46c9a9467 None None] The sync data for ids ['7ead47d0-d22c-480f-8edb-a291a66ba849'] in dv-cinder-791239-2-iscsi is: [{'id': '7ead47d0-d22c-480f-8edb-a291a66ba849', 'name': 'tempest-TestMinimumBasicScenario-1131347140-router', 'tenant_id': '4b47c7cc67c545eab92ea86cde1abac7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:49Z', 'updated_at': '2021-05-24T10:07:57Z', 'revision_number': 6, 'project_id': '4b47c7cc67c545eab92ea86cde1abac7', 'gw_port_host': None}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:58.673411 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:58.681762 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b9d49ba8-e4bd-438d-a99b-bf999da340d0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:07:58.705002 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:58.710540 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:07:58.721829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:07:58.722488 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:07:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:58.768114 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Pushing event deleted for resources: {'Port': ['ID=8c72ce53-67f4-42a6-beef-bb11cea7beb2,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:07:58.775639 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:07:58.968915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.037740 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:07:59.096808 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6f3d8b57-93eb-4552-b977-05aff498e6b3 req-1bd41ea3-89bb-4805-a91f-b8a62446518a tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/ports?device_id=812a56d8-d836-470a-b693-994765367aec HTTP/1.1" status: 200 len: 210 time: 0.2059731 May 24 10:07:59.141911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:07:59.141911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eca6df41-ad3d-4f1a-9a5c-cd20b4f97938 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:07:59.185111 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:07:59.195023 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.197883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.267842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60040) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:59.463291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-743202037-network', 'tenant_id': '4510e9fd97024c6388c942a21f7aa06c'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:07:59.469088 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:59.473833 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Usage tracker for resource:network and tenant:4510e9fd97024c6388c942a21f7aa06c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:07:59.476799 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:07:59.488071 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:59.493402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.493402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 7ead47d0-d22c-480f-8edb-a291a66ba849 {{(pid=109112) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:07:59.516944 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.519113 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-4faaf2d5-16b0-459c-b5b5-f763d04824d1 admin admin] 10.222.0.67 "DELETE /v2.0/routers/7ead47d0-d22c-480f-8edb-a291a66ba849 HTTP/1.1" status: 204 len: 168 time: 2.4316397 May 24 10:07:59.521748 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60046) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:07:59.578288 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Made reservation on behalf of 4510e9fd97024c6388c942a21f7aa06c for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:07:59.581850 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.606022 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.644230 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Deleting subnet a1ecfc88-eaf2-426e-bb90-f6a474fb5991 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:07:59.649017 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:07:59.652206 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.654557 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:07:59.663466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-429c1179-7e40-4f9b-80d1-2793f7eae1b5 None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:00:36Z', 'revision_number': 4, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': '367e9138-2d81-4a46-8056-5d07308e918d', 'name': '', 'network_id': 'c1fb75f5-deee-4236-9b04-1b6b5e617606', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'mac_address': 'fa:16:3e:0c:2f:76', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_interface', 'standard_attr_id': 91, 'fixed_ips': [{'subnet_id': '327631ed-b242-499d-854c-7f681e3bd80e', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:35Z', 'updated_at': '2021-05-24T10:00:47Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'subnets': [{'id': '327631ed-b242-499d-854c-7f681e3bd80e', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:07:59.692511 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Usage tracker for resource:security_group_rule and tenant:4510e9fd97024c6388c942a21f7aa06c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:07:59.697349 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:07:59.712024 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:07:59.731814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.871069 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:07:59.938505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.003646 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45d84c98-675a-4026-a1bf-b8d03ff7c018 None None] New status for floating IP 9453c2e9-84b1-4b24-856e-dc0745e22ee7: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:08:00.091335 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.101305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e6275947-c8e1-4d60-b630-fdb313607fab tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:00.181457 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 68} {{(pid=109113) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:08:00.221190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=135bc0f7-bbd9-4ce2-9a56-fc0e27434d7f,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:00.226215 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:00.234998 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.234998 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Added segment 723647f3-fb26-4b68-b395-80e6d592484e of type vlan for network f430632e-2cef-42d7-b127-460f46b22d5a May 24 10:08:00.317586 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.361087 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60056) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:00.522086 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8c72ce53-67f4-42a6-beef-bb11cea7beb2 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:00.548233 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.549323 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.553027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:00.553819 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.563355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.567577 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 8c72ce53-67f4-42a6-beef-bb11cea7beb2 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:00.567932 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8c72ce53-67f4-42a6-beef-bb11cea7beb2 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:00.571596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:00.631888 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a9b10b80-d568-415e-bfe9-094fa8070a8b None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:00.632245 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.714901 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/a1ecfc88-eaf2-426e-bb90-f6a474fb5991 HTTP/1.1" status: 204 len: 168 time: 1.1917841 May 24 10:08:00.718135 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60060) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:00.719533 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.727538 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Pushing event updated for resources: {'Network': ['ID=f430632e-2cef-42d7-b127-460f46b22d5a,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:00.733598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:00.734055 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.150s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:00.773446 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=a1ecfc88-eaf2-426e-bb90-f6a474fb5991,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:00.778513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5af96060-2564-4e6e-8c32-7be5bca547a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:00.783397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network f430632e-2cef-42d7-b127-460f46b22d5a details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:08:00.797866 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-133bfda2-a742-4ab8-99b5-dd8db79827be admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 896 time: 1.5253930 May 24 10:08:00.798109 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60064) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:00.804500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Request body: {'subnet': {'network_id': 'f430632e-2cef-42d7-b127-460f46b22d5a', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-743202037-subnet', 'tenant_id': '4510e9fd97024c6388c942a21f7aa06c', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:00.810723 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:00.815685 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Usage tracker for resource:subnet and tenant:4510e9fd97024c6388c942a21f7aa06c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:00.819025 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:00.827723 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:00.907806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.913900 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-be9abd95-ca5a-4abf-be29-3706862ded1b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:00.926319 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.930584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Made reservation on behalf of 4510e9fd97024c6388c942a21f7aa06c for: {'subnet': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:00.933759 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-128981363'] for subnet, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:00.951397 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=b0a9d06a-7b60-46b7-9ded-3120b0dc1680,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:00.961995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:01.019547 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Deleting port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:01.020173 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.043655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.044705 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "DELETE /v2.0/security-groups/b0a9d06a-7b60-46b7-9ded-3120b0dc1680 HTTP/1.1" status: 204 len: 168 time: 0.6826756 May 24 10:08:01.047721 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60068) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:01.068261 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.api.v2.resource [None req-e0aba4c7-989c-41ec-845c-503feda029c0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] show failed (client error): The resource could not be found. May 24 10:08:01.074962 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e0aba4c7-989c-41ec-845c-503feda029c0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "GET /v2.0/security-groups/b0a9d06a-7b60-46b7-9ded-3120b0dc1680 HTTP/1.1" status: 404 len: 347 time: 0.0208149 May 24 10:08:01.075250 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60072) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:01.150070 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] For port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:01.150583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.306239 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Calling delete_port for 4bb8a808-7920-4267-8cbc-98c3ce9c0bab owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:01.323142 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] DHCP port deletion for d34a1523-ea50-45ad-a897-459ffff7af90 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:08:01.423040 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Deleting port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:01.423574 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.439254 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Deleting port 842d1931-9818-44e9-b880-ba879c97aaa3 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:01.439855 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.461365 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.466171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-034a04d5-e9ae-4eaf-91f4-8695c4447d5d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:01.515372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Pushing event deleted for resources: {'Port': ['ID=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:01.518765 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:01.519573 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] No DVR routers for this DVR port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:01.614815 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.615112 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.651687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.652849 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] For port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,segment=NetworkSegment(d0cd8b26-954b-4e20-bc84-0cd77d9d3588),segment_id=d0cd8b26-954b-4e20-bc84-0cd77d9d3588)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:01.653785 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.669961 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['d34a1523-ea50-45ad-a897-459ffff7af90']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:01.693392 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:01.693984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'd0cd8b26-954b-4e20-bc84-0cd77d9d3588') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:01.715364 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.737655 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.746274 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Releasing vlan 82 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:08:01.746979 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:08:01.762690 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:08:01.803330 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Calling delete_port for 842d1931-9818-44e9-b880-ba879c97aaa3 owned by network:floatingip {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:01.916234 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Calling delete_port for 4bb8a808-7920-4267-8cbc-98c3ce9c0bab owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:01.924502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.938592 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:01.943826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:01.980066 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Pushing event deleted for resources: {'Network': ['ID=d34a1523-ea50-45ad-a897-459ffff7af90,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:01.984368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:01.987906 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] 10.222.0.67 "DELETE /v2.0/networks/d34a1523-ea50-45ad-a897-459ffff7af90 HTTP/1.1" status: 204 len: 168 time: 1.2665575 May 24 10:08:01.990695 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60076) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:01.997543 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:02.001769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:02.042100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Network d34a1523-ea50-45ad-a897-459ffff7af90 is not hosted by any dhcp agent {{(pid=109114) _notify_agents /opt/stack/neutron/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:231}} May 24 10:08:02.042635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] No DVR routers for this DVR port 4bb8a808-7920-4267-8cbc-98c3ce9c0bab on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:02.051065 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Pushing event deleted for resources: {'Port': ['ID=4bb8a808-7920-4267-8cbc-98c3ce9c0bab,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:02.054365 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:02.146701 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-129003893', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-128981349'] for subnet, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:02.186879 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:02.571739 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4bb8a808-7920-4267-8cbc-98c3ce9c0bab no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:02.611966 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 4bb8a808-7920-4267-8cbc-98c3ce9c0bab {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:02.612324 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 4bb8a808-7920-4267-8cbc-98c3ce9c0bab not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:02.645467 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:02.660729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Pushing event updated for resources: {'Subnet': ['ID=d373c31a-0cf6-40b8-ac6a-42d532ab6772,revision_number=0']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:02.675942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.489s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:02.679310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49c6633b-e8dd-424d-991a-52334589fcfb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:02.771225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Pushing event deleted for resources: {'Port': ['ID=842d1931-9818-44e9-b880-ba879c97aaa3,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:02.781860 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Checking segment: {'id': '723647f3-fb26-4b68-b395-80e6d592484e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 68, 'network_id': 'f430632e-2cef-42d7-b127-460f46b22d5a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:08:02.782223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Network hostable DHCP agents. Network: f430632e-2cef-42d7-b127-460f46b22d5a, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109113) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:08:02.785486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:02.996255 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks [] for floatingip, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:03.048190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:03.049617 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] 10.222.0.67 "DELETE /v2.0/floatingips/9453c2e9-84b1-4b24-856e-dc0745e22ee7 HTTP/1.1" status: 204 len: 168 time: 1.9731252 May 24 10:08:03.095317 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Network f430632e-2cef-42d7-b127-460f46b22d5a is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109113) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:08:03.201083 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, f430632e-2cef-42d7-b127-460f46b22d5a) {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:08:03.213502 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5bfeb4b6-48fb-421f-9317-ce3630c31b53 None None] Network f430632e-2cef-42d7-b127-460f46b22d5a requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:03.219036 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:03.219696 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:03.325190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:03.391790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d6058022-45b3-48eb-aaaa-909ed8a4a3de None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:03.421103 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-5d95d91f-fbb4-44d0-bbca-1591069fc0b6 admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 2.6214123 May 24 10:08:03.528817 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0fbf80c0-d862-4c3f-82a4-9b11c6e493dc None None] Network f430632e-2cef-42d7-b127-460f46b22d5a requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:03.976131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Deleting port 350a3cff-ca12-4254-a4cf-07a695eb5914 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:03.976957 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:04.158232 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] For port 350a3cff-ca12-4254-a4cf-07a695eb5914, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=350a3cff-ca12-4254-a4cf-07a695eb5914,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:04.158737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:04.316520 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Calling delete_port for 350a3cff-ca12-4254-a4cf-07a695eb5914 owned by network:router_interface {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:04.532354 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60100) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:04.540944 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-696129364-network', 'tenant_id': 'c9f5f8a373b14f4da2fdc5bac02b6682'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:04.546323 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:04.551638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Usage tracker for resource:network and tenant:c9f5f8a373b14f4da2fdc5bac02b6682 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:04.555211 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:04.562834 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:04.622265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:04.626769 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:04.794973 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Made reservation on behalf of c9f5f8a373b14f4da2fdc5bac02b6682 for: {'network': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:04.797390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:04.821034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:04.832963 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Pushing event deleted for resources: {'Port': ['ID=350a3cff-ca12-4254-a4cf-07a695eb5914,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:04.873945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.247s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:04.913797 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:04.924717 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Usage tracker for resource:security_group_rule and tenant:c9f5f8a373b14f4da2fdc5bac02b6682 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:04.929934 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:04.943452 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:05.335473 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.496451 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.637318 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.643681 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1b08b76f-3dff-48ef-aa53-1c40277bb3a4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:05.678506 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 79} {{(pid=109113) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:08:05.771465 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ca99c00b-f042-4f59-94c9-617038d299d4,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:05.774931 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:05.776335 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.777269 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.segments_db [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Added segment ee58eb3b-9c79-455d-bf65-53bcaa922de5 of type vlan for network d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb May 24 10:08:05.830879 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.862495 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a93a1c0f-ef69-4a31-b6a0-baf8772c8143 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.925039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:05.955676 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Router 66fcf049-caec-4596-8b28-a1655969d1af has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:08:06.082146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:06.099851 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:06.100859 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:06.125327 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:06.338943 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Pushing event updated for resources: {'Network': ['ID=d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:06.342399 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.217s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:06.350852 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:08:06.365311 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 896 time: 1.8313248 May 24 10:08:06.366539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:08:06.368358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60116) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:06.372643 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] 10.222.0.67 "PUT /v2.0/routers/66fcf049-caec-4596-8b28-a1655969d1af/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 4.3763063 May 24 10:08:06.375925 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60120) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:06.383402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Request body: {'subnet': {'network_id': 'd588b6c1-fb3b-4cf4-9df1-fd1f813d26fb', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-696129364-subnet', 'tenant_id': 'c9f5f8a373b14f4da2fdc5bac02b6682', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:06.389607 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Sync routers for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:08:06.390032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:06.395296 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Usage tracker for resource:subnet and tenant:c9f5f8a373b14f4da2fdc5bac02b6682 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:06.399456 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:06.407168 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:06.517691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Made reservation on behalf of c9f5f8a373b14f4da2fdc5bac02b6682 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:06.525520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:06.525719 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:08:06.525719 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:08:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:06.598677 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60128) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:06.909434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks [] for router, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:06.938540 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:08:07.304370 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.329999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.476910 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145'}], 'enable_snat': True}, 'gw_port_id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:08:04Z', 'revision_number': 5, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port': {'id': '092d3b12-88bd-4f00-a17e-3ebfeb696ec6', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:40:65:dd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '66fcf049-caec-4596-8b28-a1655969d1af', 'device_owner': 'network:router_gateway', 'standard_attr_id': 444, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.145', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:05:00Z', 'updated_at': '2021-05-24T10:05:06Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:07.482142 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Checking router: 66fcf049-caec-4596-8b28-a1655969d1af for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:08:07.492790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Routers {'66fcf049-caec-4596-8b28-a1655969d1af'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:08:07.493591 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:08:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['66fcf049-caec-4596-8b28-a1655969d1af'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:07.576550 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Deleting port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:07.577347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.697931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.707997 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:07.716635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] Set router 66fcf049-caec-4596-8b28-a1655969d1af gateway port host: None {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:08:07.799995 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] For port 092d3b12-88bd-4f00-a17e-3ebfeb696ec6, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:07.800523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.834530 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:08:07Z', 'revision_number': 6, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:07.838828 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3d87a16a-b1a5-4ff0-a2f9-de1878cae6de None None] The sync data for ids ['66fcf049-caec-4596-8b28-a1655969d1af'] in dv-cinder-791239-2-iscsi is: [{'id': '66fcf049-caec-4596-8b28-a1655969d1af', 'name': 'tempest-TestMinimumBasicScenario-2111530498-router', 'tenant_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:04:59Z', 'updated_at': '2021-05-24T10:08:07Z', 'revision_number': 6, 'project_id': 'f7673d41f8314855baa9c60fd5e2e1ba', 'gw_port_host': None}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:08:07.849874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:07.880918 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Pushing event updated for resources: {'Subnet': ['ID=9c097e57-d587-4793-924c-394f5b799822,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:07.958477 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Calling delete_port for 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 owned by network:router_gateway {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:07.968304 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Checking segment: {'id': 'ee58eb3b-9c79-455d-bf65-53bcaa922de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 79, 'network_id': 'd588b6c1-fb3b-4cf4-9df1-fd1f813d26fb'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:08:07.968707 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Network hostable DHCP agents. Network: d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:08:07.972456 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.972832 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.973223 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.973594 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.973953 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.974401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.974755 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.975118 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.975472 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.975822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.976173 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.976519 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:07.983693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.276s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:08.110514 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Network d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:08:08.210699 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:08:08.212476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.223429 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e5129e8c-f49a-4d5f-8b56-9ddf3b7edea0 None None] Network d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:08.227683 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:08.364576 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Pushing event deleted for resources: {'Port': ['ID=092d3b12-88bd-4f00-a17e-3ebfeb696ec6,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:08.378388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:08.379003 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.383064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:08.401081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Deleting port 367e9138-2d81-4a46-8056-5d07308e918d {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:08.402267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.462531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.500226 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:08:08.501810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-71132bca-d50a-422c-8eee-5ec855618ed2 None None] Network d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:08.581170 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 2.2112212 May 24 10:08:08.608912 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] For port 367e9138-2d81-4a46-8056-5d07308e918d, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=367e9138-2d81-4a46-8056-5d07308e918d,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:08.608912 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.671006 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.675343 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.767104 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60136) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:08.822307 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Calling delete_port for 367e9138-2d81-4a46-8056-5d07308e918d owned by network:router_interface {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:08.822767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:08.847829 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Deleting subnet d373c31a-0cf6-40b8-ac6a-42d532ab6772 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:08:08.852437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.921747 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.947322 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:08.948157 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 092d3b12-88bd-4f00-a17e-3ebfeb696ec6 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:08.949428 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 350a3cff-ca12-4254-a4cf-07a695eb5914 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:08.988784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:08.989356 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Fanout notify agent at l3_agent the message router_deleted on router 66fcf049-caec-4596-8b28-a1655969d1af {{(pid=109112) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:08:08.999397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.000798 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 350a3cff-ca12-4254-a4cf-07a695eb5914 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:09.001353 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 350a3cff-ca12-4254-a4cf-07a695eb5914 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:09.011624 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.039954 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.041199 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] 10.222.0.67 "DELETE /v2.0/routers/66fcf049-caec-4596-8b28-a1655969d1af HTTP/1.1" status: 204 len: 168 time: 2.6637938 May 24 10:08:09.044411 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60140) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:09.082241 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.093974 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8afcd0bb-9029-4c4d-a9dd-f94ffe05dafa tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:09.155699 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Pushing event deleted for resources: {'Port': ['ID=367e9138-2d81-4a46-8056-5d07308e918d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:09.157737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:09.201440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_v2 [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Deleting subnet ffc35f12-deef-45c7-9064-c4d2fba75cd3 {{(pid=109113) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:08:09.238739 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.309807 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128842496'] for subnet, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.485092 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.489785 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:09.664835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=d373c31a-0cf6-40b8-ac6a-42d532ab6772,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:09.664835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:09.664835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.675220 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6a4319ed-b37e-41e5-990d-41ceea687c9d admin admin] 10.222.0.67 "DELETE /v2.0/subnets/d373c31a-0cf6-40b8-ac6a-42d532ab6772 HTTP/1.1" status: 204 len: 168 time: 0.9062634 May 24 10:08:09.677382 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60146) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:09.709884 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.717694 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.796111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.937461 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:09.952116 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['f430632e-2cef-42d7-b127-460f46b22d5a']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:09.966408 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '723647f3-fb26-4b68-b395-80e6d592484e') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:09.966751 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '723647f3-fb26-4b68-b395-80e6d592484e') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:09.980086 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.000553 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.010117 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Releasing vlan 68 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:08:10.010615 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:08:10.025435 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:08:10.176262 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.181351 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:10.197171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.210402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Pushing event deleted for resources: {'Network': ['ID=f430632e-2cef-42d7-b127-460f46b22d5a,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:10.218131 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] 10.222.0.67 "DELETE /v2.0/networks/f430632e-2cef-42d7-b127-460f46b22d5a HTTP/1.1" status: 204 len: 168 time: 0.5370698 May 24 10:08:10.218131 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:10.220491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60152) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:10.293393 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Deleting subnet 9c097e57-d587-4793-924c-394f5b799822 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:08:10.296986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.358706 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.423983 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.504502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Router cae69d10-56d3-4334-8870-67746df8a973 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:08:10.643362 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-129003830', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-128981356'] for subnet, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.683501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cb6a4b56-6c24-4b68-bf66-6d5dc8c1e7d4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:10.755766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=ffc35f12-deef-45c7-9064-c4d2fba75cd3,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:10.770413 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:10.772182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] Notify callbacks [] for subnet, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.773242 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-47641435-5b23-4966-8f13-a3fcbb48e545 None None] Network a5c48f0c-499f-4abd-b04c-ed57773dbaef requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:10.822265 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.829424 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:10.867224 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=9c097e57-d587-4793-924c-394f5b799822,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:10.869593 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:10.873389 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:10.882338 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e605b0eb-0129-4e62-868d-720b7132a041 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/9c097e57-d587-4793-924c-394f5b799822 HTTP/1.1" status: 204 len: 168 time: 0.6562386 May 24 10:08:10.882772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60156) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:10.897711 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-2439517f-b3ec-433f-bf4a-47535b1acc05 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/ffc35f12-deef-45c7-9064-c4d2fba75cd3 HTTP/1.1" status: 204 len: 168 time: 1.8502610 May 24 10:08:10.905795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60160) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:10.964276 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:08:10.978390 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] 10.222.0.67 "PUT /v2.0/routers/cae69d10-56d3-4334-8870-67746df8a973/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 4.3736076 May 24 10:08:10.978390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60164) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:10.986088 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Sync routers for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:08:10.999247 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:08:11.000180 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:08:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.098888 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] DHCP port deletion for a5c48f0c-499f-4abd-b04c-ed57773dbaef request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:08:11.194388 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks [] for router, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.234607 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.235646 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:08:11.255113 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Deleting port ad6248be-368d-44d8-9aaa-9ac7d6960750 {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:11.256005 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.265008 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.472806 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] For port ad6248be-368d-44d8-9aaa-9ac7d6960750, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ad6248be-368d-44d8-9aaa-9ac7d6960750,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:11.473228 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.556201 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:08:08Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.561230 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Checking router: cae69d10-56d3-4334-8870-67746df8a973 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:08:11.569373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.571665 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Routers {'cae69d10-56d3-4334-8870-67746df8a973'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:08:11.572506 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:08:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['cae69d10-56d3-4334-8870-67746df8a973'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.586692 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.602508 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.605465 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'ee58eb3b-9c79-455d-bf65-53bcaa922de5') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.605881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'ee58eb3b-9c79-455d-bf65-53bcaa922de5') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:11.624531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.648796 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Deleting port ad6248be-368d-44d8-9aaa-9ac7d6960750 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:11.649326 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.663516 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.685362 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Releasing vlan 79 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:08:11.685875 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:08:11.700277 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:08:11.763434 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Calling delete_port for ad6248be-368d-44d8-9aaa-9ac7d6960750 owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:11.795274 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] Set router cae69d10-56d3-4334-8870-67746df8a973 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:08:11.912738 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Deleting port 6f0c5b38-17fc-465c-926f-712944583a0a {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:11.912738 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:11.964489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] For port ad6248be-368d-44d8-9aaa-9ac7d6960750, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=ad6248be-368d-44d8-9aaa-9ac7d6960750,segment=NetworkSegment(62003036-d603-4e95-8128-f4d87aace384),segment_id=62003036-d603-4e95-8128-f4d87aace384)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:11.965464 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.003071 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.010476 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:12.090918 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] For port 6f0c5b38-17fc-465c-926f-712944583a0a, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=6f0c5b38-17fc-465c-926f-712944583a0a,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:12.091422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.101903 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Pushing event deleted for resources: {'Port': ['ID=ad6248be-368d-44d8-9aaa-9ac7d6960750,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:12.107945 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:12.118514 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] No DVR routers for this DVR port ad6248be-368d-44d8-9aaa-9ac7d6960750 on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:12.122976 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.138566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Calling delete_port for ad6248be-368d-44d8-9aaa-9ac7d6960750 owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:12.138566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:12.152375 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:08:08Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:12.153631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.159522 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7efc0f18-5771-4899-a1a9-daafb74e5185 None None] The sync data for ids ['cae69d10-56d3-4334-8870-67746df8a973'] in dv-cinder-791239-2-iscsi is: [{'id': 'cae69d10-56d3-4334-8870-67746df8a973', 'name': 'tempest-AttachVolumeShelveTestJSON-1535052560-router', 'tenant_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34'}], 'enable_snat': True}, 'gw_port_id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:00:27Z', 'updated_at': '2021-05-24T10:08:08Z', 'revision_number': 5, 'project_id': 'c42fc1bb2ae6477fa7b7c9261f2d50bb', 'gw_port': {'id': '6f0c5b38-17fc-465c-926f-712944583a0a', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:ab:8e:e1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cae69d10-56d3-4334-8870-67746df8a973', 'device_owner': 'network:router_gateway', 'standard_attr_id': 65, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.34', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:00:29Z', 'updated_at': '2021-05-24T10:00:39Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:08:12.166816 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Pushing event deleted for resources: {'Network': ['ID=d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:12.171143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:12.177333 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ab4a558d-14dd-4351-8fbc-5fbea6ec1be9 admin admin] 10.222.0.67 "DELETE /v2.0/networks/d588b6c1-fb3b-4cf4-9df1-fd1f813d26fb HTTP/1.1" status: 204 len: 168 time: 1.2887063 May 24 10:08:12.248000 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.255186 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-08fb96a0-3dcc-4de3-bdfa-420e0f1a0b97 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:12.272488 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Calling delete_port for 6f0c5b38-17fc-465c-926f-712944583a0a owned by network:router_gateway {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:12.304131 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Pushing event deleted for resources: {'Port': ['ID=ad6248be-368d-44d8-9aaa-9ac7d6960750,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:12.306509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] No DVR routers for this DVR port ad6248be-368d-44d8-9aaa-9ac7d6960750 on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:12.308588 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:12.432716 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.450738 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['a5c48f0c-499f-4abd-b04c-ed57773dbaef']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:12.476110 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '62003036-d603-4e95-8128-f4d87aace384') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:12.476759 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '62003036-d603-4e95-8128-f4d87aace384') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:12.497993 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.519464 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.531553 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Releasing vlan 97 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:08:12.532068 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ml2_port_bindings' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: util.warn( May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ml2_port_binding_levels' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: util.warn( May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'portsecuritybindings' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: util.warn( May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ports' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: util.warn( May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:08:12.558420 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:08:12.596544 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60170) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:12.624236 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.630968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-612c7a2f-a1ae-4857-a4c1-ce2905576d3e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:12.656768 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 367e9138-2d81-4a46-8056-5d07308e918d no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:12.697999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Pushing event deleted for resources: {'Port': ['ID=6f0c5b38-17fc-465c-926f-712944583a0a,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:12.722776 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 367e9138-2d81-4a46-8056-5d07308e918d {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:12.723394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 367e9138-2d81-4a46-8056-5d07308e918d not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:12.724022 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ad6248be-368d-44d8-9aaa-9ac7d6960750 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:12.746470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:12.792331 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-eac8492b-a553-4ff1-9e05-073e956750ee admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=4510e9fd97024c6388c942a21f7aa06c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1910357 May 24 10:08:12.793585 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60174) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:12.824559 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: ad6248be-368d-44d8-9aaa-9ac7d6960750 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:12.824559 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device ad6248be-368d-44d8-9aaa-9ac7d6960750 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:12.831335 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.862524 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.872409 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:12.894051 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.898542 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Pushing event deleted for resources: {'Network': ['ID=a5c48f0c-499f-4abd-b04c-ed57773dbaef,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:12.907596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:12.912702 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ad9df0b8-5ce6-42f9-8095-cc72e0eed503 admin admin] 10.222.0.67 "DELETE /v2.0/networks/a5c48f0c-499f-4abd-b04c-ed57773dbaef HTTP/1.1" status: 204 len: 168 time: 2.0057716 May 24 10:08:12.932451 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:12.986613 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.042816 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.045750 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.170265 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.174424 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9d70a6fe-8266-4f4c-833b-9c96676db623 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:13.187951 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=135bc0f7-bbd9-4ce2-9a56-fc0e27434d7f,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:13.192005 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:13.196052 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.197284 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-617f257a-c5b1-419f-aab0-a8173bc95cad admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/135bc0f7-bbd9-4ce2-9a56-fc0e27434d7f HTTP/1.1" status: 204 len: 168 time: 0.4020603 May 24 10:08:13.208347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.210788 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Fanout notify agent at l3_agent the message router_deleted on router cae69d10-56d3-4334-8870-67746df8a973 {{(pid=109113) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:08:13.237890 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.239194 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] 10.222.0.67 "DELETE /v2.0/routers/cae69d10-56d3-4334-8870-67746df8a973 HTTP/1.1" status: 204 len: 168 time: 2.2601867 May 24 10:08:13.243503 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60182) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:13.320432 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Deleting subnet 327631ed-b242-499d-854c-7f681e3bd80e {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:08:13.324696 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.395733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.503833 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60186) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:13.528806 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.561403 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-4a310b0f-148c-4bef-95f2-781a529a8faf admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=4b47c7cc67c545eab92ea86cde1abac7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0562615 May 24 10:08:13.566252 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60190) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:13.668785 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.718856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.933791 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.938070 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d76c5fa7-0369-4e8a-a520-a842630c5405 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:13.954115 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e83316d5-9d97-4e86-bce6-f8bde9c85f77,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:13.957125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:13.959646 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:13.964201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1f1fe125-4ed3-4c9e-9eca-586456b56c89 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:14.007034 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=327631ed-b242-499d-854c-7f681e3bd80e,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:14.008991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:14.010025 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.016748 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-31d25841-264f-4359-9414-c8773dfc3d50 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/327631ed-b242-499d-854c-7f681e3bd80e HTTP/1.1" status: 204 len: 168 time: 0.7721386 May 24 10:08:14.017202 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3f544aa5-552c-4d6d-828c-5cc86f0147b7 None None] Network c1fb75f5-deee-4236-9b04-1b6b5e617606 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:14.021573 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60196) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:14.063063 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.064076 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/e83316d5-9d97-4e86-bce6-f8bde9c85f77 HTTP/1.1" status: 204 len: 168 time: 0.4964395 May 24 10:08:14.169951 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.251957 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60202) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:14.318047 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Deleting port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:14.318047 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.325123 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-4d0327ae-7b5c-4efd-9e83-9b5269eac20a admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c9f5f8a373b14f4da2fdc5bac02b6682&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0701754 May 24 10:08:14.328246 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60206) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:14.434376 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] DHCP port deletion for c1fb75f5-deee-4236-9b04-1b6b5e617606 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:08:14.467373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.496874 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] For port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:14.497480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.525147 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.564404 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Deleting port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:08:14.566832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.674883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 6f0c5b38-17fc-465c-926f-712944583a0a no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:14.732229 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 6f0c5b38-17fc-465c-926f-712944583a0a {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:08:14.732753 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 6f0c5b38-17fc-465c-926f-712944583a0a not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:14.733221 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 3e6b3d69-2ada-4c22-8efb-d8aef48472ab no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:08:14.787290 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] For port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,segment=NetworkSegment(f1abdab4-bf27-4888-b131-574a25f02d59),segment_id=f1abdab4-bf27-4888-b131-574a25f02d59)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:14.789766 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.789959 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.792172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ff757ee5-0c6a-4d8c-bbba-04cd6f1e45c4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:14.810610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ca99c00b-f042-4f59-94c9-617038d299d4,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:14.813902 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:14.822983 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:14.826368 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-63991faa-b8d0-4d45-88e1-9853c64e2b83 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/ca99c00b-f042-4f59-94c9-617038d299d4 HTTP/1.1" status: 204 len: 168 time: 0.4943299 May 24 10:08:14.872508 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Calling delete_port for 3e6b3d69-2ada-4c22-8efb-d8aef48472ab owned by network:dhcp {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:15.006717 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Calling delete_port for 3e6b3d69-2ada-4c22-8efb-d8aef48472ab owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:08:15.019720 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.020047 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No active binding found for port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab {{(pid=109114) get_port_binding_host /opt/stack/neutron/neutron/plugins/ml2/db.py:224}} May 24 10:08:15.020734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 3e6b3d69-2ada-4c22-8efb-d8aef48472ab not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:08:15.026806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-fa1a0f65-e01b-4991-8ac5-71516fc405e0 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:15.082294 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Pushing event deleted for resources: {'Port': ['ID=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:15.083255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.085371 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:15.086176 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] No DVR routers for this DVR port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab on host dv-cinder-791239-2-iscsi {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:15.093995 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-d1231fbf-4c35-4564-9ae0-c6c7672f7d22 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:15.191779 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Pushing event deleted for resources: {'Port': ['ID=3e6b3d69-2ada-4c22-8efb-d8aef48472ab,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:15.196777 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:15.198233 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] No DVR routers for this DVR port 3e6b3d69-2ada-4c22-8efb-d8aef48472ab on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:08:15.225853 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.245238 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c1fb75f5-deee-4236-9b04-1b6b5e617606']}} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:15.260004 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f1abdab4-bf27-4888-b131-574a25f02d59') {'for_net_delete': True} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:15.260310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.services.segments.db [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f1abdab4-bf27-4888-b131-574a25f02d59') {} {{(pid=109113) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:08:15.274380 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128842507'] for segment, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.294245 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.309352 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Releasing vlan 90 on physical network physnet1 to pool {{(pid=109113) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:08:15.310019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-129003664'] for segment, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.317356 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60216) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: Traceback (most recent call last): May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: timer() May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: cb(*args, **kw) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: return func(*args, **kwargs) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: self._notify() May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: self.callback(batched_events) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: event.method(event) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: return f(self, *a, **k) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._get(url).json() May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: return self.request(url, 'GET', **kwargs) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:08:15.332621 dv-cinder-791239-2-iscsi neutron-server[109113]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:08:15.402376 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f4af37ec-19c0-489b-80d0-7daa30337b71 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=f7673d41f8314855baa9c60fd5e2e1ba&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0817285 May 24 10:08:15.405170 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60222) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:15.483489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.516387 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.524354 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.544402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:15.564680 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.585224 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Pushing event deleted for resources: {'Network': ['ID=c1fb75f5-deee-4236-9b04-1b6b5e617606,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:15.594983 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:15.678828 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.683650 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-1c3c74f9-46c0-4b8c-b5d5-ef5990cc4c4c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:15.697907 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f369c076-92b5-438f-b571-9bbb565f2aae,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:15.701295 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:15.707702 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:15.709511 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3bae9d2a-8c4b-4a81-8817-deeccf199c6e admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/f369c076-92b5-438f-b571-9bbb565f2aae HTTP/1.1" status: 204 len: 168 time: 0.3028903 May 24 10:08:15.725077 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] 10.222.0.67 "DELETE /v2.0/networks/c1fb75f5-deee-4236-9b04-1b6b5e617606 HTTP/1.1" status: 204 len: 168 time: 1.7014871 May 24 10:08:16.262394 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60236) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:16.321011 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-204a785b-9dfb-474f-adde-ecfc90363c78 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c42fc1bb2ae6477fa7b7c9261f2d50bb&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0544689 May 24 10:08:16.322101 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60242) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:16.420722 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:16.467202 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:16.627259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:16.631727 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b0a00c74-9be7-4658-8433-ac36eb1f7acc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:16.647607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5779f0cc-6cbf-464c-be65-041f64dd4bc4,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:16.651697 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:16.658619 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:16.660950 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/5779f0cc-6cbf-464c-be65-041f64dd4bc4 HTTP/1.1" status: 204 len: 168 time: 0.3377635 May 24 10:08:17.981010 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60284) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:18.326353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeNegativeTest-1264842732-network', 'tenant_id': '8e85a9260aad4c93aa21262f8764f011'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:18.333372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:18.343358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Usage tracker for resource:network and tenant:8e85a9260aad4c93aa21262f8764f011 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:18.347523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:18.355730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:18.438825 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:18.441793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:18.455487 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:18.471353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:18.476212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Usage tracker for resource:security_group_rule and tenant:8e85a9260aad4c93aa21262f8764f011 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:18.483850 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:18.493321 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:18.600805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:18.701598 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:18.711426 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7811f260-454b-4c0f-80dc-58d0696638f9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:18.742461 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b7676221-dbea-47ff-b118-72910e441b53,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:18.746647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:18.833212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 76} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:08:18.850596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:18.851244 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Added segment b42a154a-6873-4e20-9753-b2afc6ed11b0 of type vlan for network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd May 24 10:08:18.890414 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:19.004357 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:19.006079 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:19.006758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:19.011903 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0ec8f92e-f8a6-4622-b0db-94fef40fa1e2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:19.124868 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Pushing event updated for resources: {'Network': ['ID=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:19.128345 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:19.137294 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:08:19.211122 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 1.2287366 May 24 10:08:19.214833 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60302) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:08:19.222220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Request body: {'subnet': {'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeNegativeTest-1264842732-subnet', 'tenant_id': '8e85a9260aad4c93aa21262f8764f011', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:19.228335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:19.235352 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Usage tracker for resource:subnet and tenant:8e85a9260aad4c93aa21262f8764f011 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:19.239089 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:19.247752 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:19.331072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:19.336761 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:19.968034 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:19.972096 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:20.050608 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Pushing event updated for resources: {'Subnet': ['ID=60925914-ac92-409e-94f9-b096645b63f1,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:20.056429 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:20.065955 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Checking segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:08:20.066308 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Network hostable DHCP agents. Network: 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:08:20.187038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:08:20.198253 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:20.255982 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:08:20.268195 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a47a0820-5860-4d4e-a7e5-8cda509ac421 None None] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:20.269210 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:20.269984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:20.370687 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-46184859-a923-40db-9731-b6b4708bbfbd admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 852 time: 1.1439297 May 24 10:08:20.498605 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-73015553-c632-4342-a8fa-56f4771e325e None None] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:08:21.060526 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-656b981b-bea3-4749-ab80-8a71d7691056 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:21.063126 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-656b981b-bea3-4749-ab80-8a71d7691056 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.1983938 May 24 10:08:21.084918 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-ab3e0253-2c5f-4347-a769-02cca83f358a tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/quotas/8e85a9260aad4c93aa21262f8764f011 HTTP/1.1" status: 200 len: 375 time: 0.0158632 May 24 10:08:21.173424 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-edc2f07c-3459-4b66-87c7-02ecab7463f5 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0822079 May 24 10:08:21.400845 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-e1f80b5e-1789-4844-8d59-c1b06e4e1b1d service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2201331 May 24 10:08:21.587844 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8db8a79c-284a-497a-860b-6159587072d0 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1791499 May 24 10:08:21.964051 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d548c48c-a397-47e6-a54b-e7704e8c56e9 req-07622807-1a05-45cb-8d33-1d11383a7bc1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.0552495 May 24 10:08:23.129262 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-43de428f-074b-457f-a454-b97c56bfe63c req-6ec87c1f-2377-456f-87c2-3365a4fac4e6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.0448048 May 24 10:08:24.300774 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-15a6e7bd-9ead-4750-b8d5-2ec5d187d16a req-2cd69847-35d1-4888-b5f9-66fc3b0af421 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.0445979 May 24 10:08:25.500048 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c3f8295c-c1bb-4568-910b-0414eda43c01 req-48f1e00f-c25f-43bf-bdab-d53547ceca80 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.0767412 May 24 10:08:26.667610 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-31bc9250-b431-4de3-87f3-6ebe5c2e8dbf req-edff4c07-b6cf-4f82-b202-cc68c3da6be6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.0446408 May 24 10:08:27.394587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-d68d02f7-96e0-4227-9bb6-08f82fc74687 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:27.396170 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-d68d02f7-96e0-4227-9bb6-08f82fc74687 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.2510202 May 24 10:08:27.409665 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Request body: {'port': {'device_id': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'admin_state_up': True, 'tenant_id': '8e85a9260aad4c93aa21262f8764f011'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:27.570143 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:27.576677 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Usage tracker for resource:port and tenant:8e85a9260aad4c93aa21262f8764f011 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:08:27.581893 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Quota usage for port was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:08:27.591736 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:27.736393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:27.742941 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:27.947155 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1480c52f-5acb-4a02-ae63-847738aa9770 req-42e341f9-eafd-4e92-877f-30f8e992a98c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.1006844 May 24 10:08:28.196714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Allocated IP 172.20.0.2 (52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd/60925914-ac92-409e-94f9-b096645b63f1/3768267d-8ff1-4093-bb55-85e4456cd30d) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:08:28.455840 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:28.633261 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:28.642127 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-91336be1-b7fb-4b93-8644-657a8c87daa4 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:28.732904 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:08:28.736602 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:08:28.744319 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:08:28.808090 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event updated for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:28.816273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.174s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:28.865668 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:28.866669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:29.056319 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.6506875 May 24 10:08:29.070826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Request body: {'port': {'device_id': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:29.189400 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f555d222-fc0c-4846-981d-ea243a2b179c req-4e7a5cd0-66dc-4f82-a988-b513b7fca423 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:29.193479 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f555d222-fc0c-4846-981d-ea243a2b179c req-4e7a5cd0-66dc-4f82-a988-b513b7fca423 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 931 time: 0.1119103 May 24 10:08:29.237383 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:29.266120 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-f555d222-fc0c-4846-981d-ea243a2b179c req-c44b7b54-ba6b-464f-b602-ab5f5cfd0603 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:08:29.416478 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-f555d222-fc0c-4846-981d-ea243a2b179c req-c44b7b54-ba6b-464f-b602-ab5f5cfd0603 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:08:29.423082 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f555d222-fc0c-4846-981d-ea243a2b179c req-c44b7b54-ba6b-464f-b602-ab5f5cfd0603 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2201319 May 24 10:08:29.718975 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 3768267d-8ff1-4093-bb55-85e4456cd30d {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:08:29.727051 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:29.746988 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Transition to ACTIVE for port object 3768267d-8ff1-4093-bb55-85e4456cd30d will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:08:29.867039 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:29.871352 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5cbeda28-e270-4ea1-9cfe-bd27f2e88b26 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:30.008796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Pushing event updated for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:30.013265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:30.123891 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Attempting to bind port 3768267d-8ff1-4093-bb55-85e4456cd30d on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:08:30.124229 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Attempting to bind port 3768267d-8ff1-4093-bb55-85e4456cd30d by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:08:30.124559 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Attempting to bind port 3768267d-8ff1-4093-bb55-85e4456cd30d on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:08:30.130093 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Port 3768267d-8ff1-4093-bb55-85e4456cd30d on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:08:30.130416 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Attempting to bind port 3768267d-8ff1-4093-bb55-85e4456cd30d on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:08:30.137016 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 8, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:08:30.137386 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Checking segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:08:30.137777 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Bound using segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:08:30.138222 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Bound port: 3768267d-8ff1-4093-bb55-85e4456cd30d, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:08:30.138575 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:30.237322 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:08:30.240931 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3768267d-8ff1-4093-bb55-85e4456cd30d,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:08:30.308081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:30.312966 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1035bb15-7f29-4cea-ab55-f292f25bb397 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:30.375397 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:30.376017 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:30.377486 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] 10.222.0.95 "PUT /v2.0/ports/3768267d-8ff1-4093-bb55-85e4456cd30d HTTP/1.1" status: 200 len: 1093 time: 1.3126378 May 24 10:08:30.378057 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:08:30.404015 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Pushing event updated for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:30.407430 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:30.440059 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d', 'status': 'completed', 'code': 200} May 24 10:08:30.584113 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-f62afa0e-3d80-40f2-a541-d43adf45d484 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:30.585969 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-f62afa0e-3d80-40f2-a541-d43adf45d484 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 1096 time: 0.1229284 May 24 10:08:30.749243 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5f64d8a-f1c9-4928-95dd-a3ffb241efc1 req-f7a24146-a7ae-4361-9c8e-2e17e03e8557 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:30.755725 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a5f64d8a-f1c9-4928-95dd-a3ffb241efc1 req-f7a24146-a7ae-4361-9c8e-2e17e03e8557 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 943 time: 0.1610377 May 24 10:08:30.785499 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-10e5feca-28c1-48c4-a893-13e0e1199fc2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:30.788711 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-10e5feca-28c1-48c4-a893-13e0e1199fc2 service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=8e85a9260aad4c93aa21262f8764f011&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1943550 May 24 10:08:30.816969 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a5f64d8a-f1c9-4928-95dd-a3ffb241efc1 req-747be7b2-20af-419c-96c3-df9ad4dca4fe tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0532749 May 24 10:08:31.077546 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-30671bff-dd34-4893-ace9-7742495d3954 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:31.078954 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fd70af56-9934-45f3-ad97-c6477ff97927 req-30671bff-dd34-4893-ace9-7742495d3954 service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.2832909 May 24 10:08:31.209514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-f0cef6d9-a460-416c-bbd5-5f9a36220f13 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:31.212070 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-f0cef6d9-a460-416c-bbd5-5f9a36220f13 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 1096 time: 0.0888855 May 24 10:08:31.299201 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-e2b40cee-2a29-4991-bb46-004daf4d02a1 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=3768267d-8ff1-4093-bb55-85e4456cd30d HTTP/1.1" status: 200 len: 217 time: 0.0783513 May 24 10:08:31.368037 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-124a820b-1a22-4871-9825-ab746234d9ab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:31.369577 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-124a820b-1a22-4871-9825-ab746234d9ab service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0642090 May 24 10:08:31.419991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-eefb3e05-82f2-4cfd-a5bc-479f27d9710c service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0440116 May 24 10:08:31.576117 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-88de6ba2-5b3a-4f1f-a59b-32bb21c2dc05 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1491289 May 24 10:08:31.726311 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cf2978a0-14ec-4db1-9189-2cddc771e9ea req-395c0172-ce1b-43bd-b1dd-a38402ff8b62 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1420987 May 24 10:08:32.044120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45336318-af2f-4b00-bb66-84423d7867eb req-0c959b46-bf54-48b1-bd52-4095a2998fed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:32.045233 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-45336318-af2f-4b00-bb66-84423d7867eb req-0c959b46-bf54-48b1-bd52-4095a2998fed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:08:32.129653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-45336318-af2f-4b00-bb66-84423d7867eb req-0c959b46-bf54-48b1-bd52-4095a2998fed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:08:32.131083 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45336318-af2f-4b00-bb66-84423d7867eb req-0c959b46-bf54-48b1-bd52-4095a2998fed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 943 time: 0.1796825 May 24 10:08:32.181388 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45336318-af2f-4b00-bb66-84423d7867eb req-b146d15d-eb15-4135-a6a9-0119879e864c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0440941 May 24 10:08:33.323994 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a0a92c1e-637f-4b60-8461-e7f74118a971 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:33.384519 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ce3ac36-f89e-48ea-95ec-1ddeba4beb58 req-a184fff1-ccfb-4450-9509-f2e8539881b9 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:33.386224 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1ce3ac36-f89e-48ea-95ec-1ddeba4beb58 req-a184fff1-ccfb-4450-9509-f2e8539881b9 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 943 time: 0.0927815 May 24 10:08:33.438498 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-1ce3ac36-f89e-48ea-95ec-1ddeba4beb58 req-f51cc7dd-ae31-4b50-93d4-cfc2fc19b5fa tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0457456 May 24 10:08:34.186419 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 3768267d-8ff1-4093-bb55-85e4456cd30d details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:08:34.331617 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3768267d-8ff1-4093-bb55-85e4456cd30d,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:34.390058 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '3768267d-8ff1-4093-bb55-85e4456cd30d', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'port_id': '3768267d-8ff1-4093-bb55-85e4456cd30d', 'mac_address': 'fa:16:3e:46:ea:b7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 76, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '60925914-ac92-409e-94f9-b096645b63f1', 'ip_address': '172.20.0.2'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:08:34.451442 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:34.480276 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 3768267d-8ff1-4093-bb55-85e4456cd30d {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:08:34.499745 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3768267d-8ff1-4093-bb55-85e4456cd30d,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:34.598368 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:34.600474 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:34.680337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2e273c3-6dc8-458a-b292-80b71c6fbfaa req-3938d25e-7d58-49a9-9b80-ac2762745123 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:34.682015 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a2e273c3-6dc8-458a-b292-80b71c6fbfaa req-3938d25e-7d58-49a9-9b80-ac2762745123 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 944 time: 0.1002176 May 24 10:08:34.685150 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:34.691059 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:34.733968 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a2e273c3-6dc8-458a-b292-80b71c6fbfaa req-ba3eedc3-e32c-4f68-86c9-fae313b99f9a tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0453415 May 24 10:08:34.818899 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:08:35.867469 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-24e3c8c7-1f20-408c-8104-a73e71580ba2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:35.935547 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:35.976979 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76906197-ff91-4518-acba-eb667a066899 req-e24bb553-9eda-4616-9997-1afa544f4a8c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:35.978765 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-76906197-ff91-4518-acba-eb667a066899 req-e24bb553-9eda-4616-9997-1afa544f4a8c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 944 time: 0.1011887 May 24 10:08:36.029812 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-76906197-ff91-4518-acba-eb667a066899 req-85335b1a-ca88-459e-bd30-c5930087ff4c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446889 May 24 10:08:36.218293 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 3768267d-8ff1-4093-bb55-85e4456cd30d up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:08:36.396253 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 3768267d-8ff1-4093-bb55-85e4456cd30d completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:08:36.401812 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 3768267d-8ff1-4093-bb55-85e4456cd30d triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:08:36.402279 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:36.554353 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:36.583770 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:08:36.593915 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3768267d-8ff1-4093-bb55-85e4456cd30d,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:08:36.634010 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d', 'code': 200} May 24 10:08:36.684969 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:36.688296 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:36.758090 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:36.762489 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:37.097081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5243d1d0-a768-41c5-bd1f-03b7f9ae9de9 req-d8c68845-631a-43d2-b1de-89156b96d8a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:37.098606 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5243d1d0-a768-41c5-bd1f-03b7f9ae9de9 req-d8c68845-631a-43d2-b1de-89156b96d8a4 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 1098 time: 0.0777264 May 24 10:08:37.158902 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5243d1d0-a768-41c5-bd1f-03b7f9ae9de9 req-573ec0b9-ebb6-4cff-ba74-c07d40650ac6 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0530121 May 24 10:08:37.209546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:37.295928 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b629c0c8-e216-4f4d-af69-7c896d90c15c req-54428ff2-dcac-4018-95a4-14fd25eb07bf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:37.297571 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b629c0c8-e216-4f4d-af69-7c896d90c15c req-54428ff2-dcac-4018-95a4-14fd25eb07bf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.0895815 May 24 10:08:37.353724 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b629c0c8-e216-4f4d-af69-7c896d90c15c req-312c11ff-40bf-4322-adf3-3d3ffecb74eb tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0501690 May 24 10:08:38.621489 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32330545-f7cf-4ee3-9aae-6c957989886b req-edc996e6-728c-457f-8d34-7a0486f1cb36 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:38.625586 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-32330545-f7cf-4ee3-9aae-6c957989886b req-edc996e6-728c-457f-8d34-7a0486f1cb36 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.1528511 May 24 10:08:38.675926 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-32330545-f7cf-4ee3-9aae-6c957989886b req-fe842441-5416-4385-bd55-8d24dde4a4b3 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0462480 May 24 10:08:38.821403 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:45.491224 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:08:50.203978 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:53.517248 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-718a9059-9450-4322-8cb1-f55945ebbfe6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:53.518797 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-718a9059-9450-4322-8cb1-f55945ebbfe6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.1668460 May 24 10:08:53.535456 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-81e7da40-b1e2-4052-8bc7-1bc95826fdd9 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/quotas/8e85a9260aad4c93aa21262f8764f011 HTTP/1.1" status: 200 len: 375 time: 0.0106502 May 24 10:08:53.627025 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-49239ce2-2914-4c80-9115-06776a8ea955 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&fields=id HTTP/1.1" status: 200 len: 255 time: 0.0813930 May 24 10:08:53.783145 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-a5fbb7e7-17d1-4cd2-967e-43819e1787d9 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1481616 May 24 10:08:53.930641 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-01c71611-2b1d-4d63-92fd-b8dd07c04d51 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1357002 May 24 10:08:54.263310 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e5d4e2c9-e405-4eb7-a2c0-ae79ea4f6c68 req-29caca65-841b-48d7-adbe-5ff063b4f0df tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 210 time: 0.0555532 May 24 10:08:55.468198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1b3eecb3-348f-4de3-a178-ffa7f3f664bd req-dbb6048e-dbf9-4a20-bd55-add2c876da4d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 210 time: 0.0509753 May 24 10:08:56.655569 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-027c5d53-593a-4ff7-961a-92fc5e328a88 req-16af6ce3-d604-42bd-8f80-4e54a25204e1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 210 time: 0.0503645 May 24 10:08:57.043304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-799ab1cb-fda2-47c5-bfae-1f954ae430dc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:57.044988 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-799ab1cb-fda2-47c5-bfae-1f954ae430dc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.2737539 May 24 10:08:57.059391 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Request body: {'port': {'device_id': '39724a92-c826-4944-8a07-916d46da0e0d', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'admin_state_up': True, 'tenant_id': '8e85a9260aad4c93aa21262f8764f011'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:57.218224 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:08:57.229149 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:08:57.295373 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:08:57.299689 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:57.487905 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-d5654af4-b085-450d-86e9-eebec38155d6 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0845659 May 24 10:08:57.503105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Allocated IP 172.20.0.9 (52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd/60925914-ac92-409e-94f9-b096645b63f1/8868b74e-802d-457a-bde2-4bd4e630837c) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:08:57.690827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-af4cb70e-4101-46cb-876e-ebb19fc5ef10 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:57.693007 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-af4cb70e-4101-46cb-876e-ebb19fc5ef10 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.0902414 May 24 10:08:57.778231 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:57.828459 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47f49ace-3d02-44af-b36b-ef691ff96a31 req-79afa779-f511-4bb5-a0f7-65596ff7f763 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 210 time: 0.0545857 May 24 10:08:57.871119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-4b9912bd-f3f6-4ff3-aeda-2221f6b1a56a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:57.872588 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-4b9912bd-f3f6-4ff3-aeda-2221f6b1a56a service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.1544418 May 24 10:08:57.895433 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:57.902697 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0b19eb1a-d9e8-4f89-9a4b-37da53590245 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:57.909349 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:08:57.988830 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-dd3c337a-62d6-4461-97d4-18fceb482ed9 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 1000 time: 0.1073048 May 24 10:08:58.043473 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:08:58.050416 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:08:58.058540 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:08:58.066048 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-cdc384fa-185d-4688-b73a-3716e8dad957 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:58.068524 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-cdc384fa-185d-4688-b73a-3716e8dad957 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.0722888 May 24 10:08:58.093936 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event updated for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:58.112079 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.112455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.112826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.113219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.113582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.113946 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.114306 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.114667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.115052 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.115388 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.115743 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.116132 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:08:58.122894 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:58.153943 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:58.154595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:58.202907 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-29b73356-253d-4fee-a9df-63765662bb00 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:58.205633 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-29b73356-253d-4fee-a9df-63765662bb00 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1289201 May 24 10:08:58.302162 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.2469311 May 24 10:08:58.315971 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Request body: {'port': {'device_id': '39724a92-c826-4944-8a07-916d46da0e0d', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:08:58.453380 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:58.456749 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-db8d1464-7f49-4e0b-b0b4-3d8d93d825ab service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2422214 May 24 10:08:58.608336 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4ddf6f8b-29e8-4d85-abc9-e679efdbfb3f req-4bd36de4-6fbc-469e-af7b-d8286df4aeea service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1437306 May 24 10:08:58.954933 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 8868b74e-802d-457a-bde2-4bd4e630837c {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:08:58.962897 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:58.981224 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Transition to ACTIVE for port object 8868b74e-802d-457a-bde2-4bd4e630837c will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:08:59.072066 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:59.077806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8c835859-4498-49e0-8561-85a8c00399cc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:59.123768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dcf39f64-eefc-44af-8049-55d1e70bf1a0 req-033634f4-86d7-45d1-92cc-be3b0eaa7edc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:59.126490 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dcf39f64-eefc-44af-8049-55d1e70bf1a0 req-033634f4-86d7-45d1-92cc-be3b0eaa7edc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 931 time: 0.1402442 May 24 10:08:59.204885 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-dcf39f64-eefc-44af-8049-55d1e70bf1a0 req-0cbdaad9-e50d-4e2f-a21d-3d02d590e488 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0711560 May 24 10:08:59.251835 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Pushing event updated for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:59.255037 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:59.356315 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Attempting to bind port 8868b74e-802d-457a-bde2-4bd4e630837c on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:08:59.356838 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Attempting to bind port 8868b74e-802d-457a-bde2-4bd4e630837c by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:08:59.357307 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Attempting to bind port 8868b74e-802d-457a-bde2-4bd4e630837c on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:08:59.363727 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Port 8868b74e-802d-457a-bde2-4bd4e630837c on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:08:59.364147 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Attempting to bind port 8868b74e-802d-457a-bde2-4bd4e630837c on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:08:59.372412 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 8, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:08:59.372895 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Checking segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:08:59.373424 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Bound using segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:08:59.374094 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Bound port: 8868b74e-802d-457a-bde2-4bd4e630837c, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:08:59.374594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:59.480753 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:08:59.484187 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=8868b74e-802d-457a-bde2-4bd4e630837c,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:08:59.569842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:59.575403 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-fc9d5ae9-7dd4-4d89-a9dd-cfc4b42b57c6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:08:59.637328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:59.637952 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:08:59.639403 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] 10.222.0.95 "PUT /v2.0/ports/8868b74e-802d-457a-bde2-4bd4e630837c HTTP/1.1" status: 200 len: 1093 time: 1.3282797 May 24 10:08:59.639942 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:08:59.669535 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Pushing event updated for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:08:59.672984 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:08:59.708386 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c', 'status': 'completed', 'code': 200} May 24 10:08:59.815954 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-cbcbffe8-de43-4b68-96f9-04330c0209c0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:08:59.817654 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-cbcbffe8-de43-4b68-96f9-04330c0209c0 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 1096 time: 0.0839202 May 24 10:09:00.107092 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-bef21a36-9945-44ee-866f-3353c33d8b00 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:00.108478 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-bef21a36-9945-44ee-866f-3353c33d8b00 service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=8e85a9260aad4c93aa21262f8764f011&shared=False HTTP/1.1" status: 200 len: 915 time: 0.2770684 May 24 10:09:00.269829 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-f7bcebd9-9323-4cec-99b5-b8ab6c52faa2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:00.271303 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b63fd001-e953-4f44-bfd0-57f6ccee6056 req-f7bcebd9-9323-4cec-99b5-b8ab6c52faa2 service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.1539404 May 24 10:09:00.394656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-6aaba74b-e12e-4917-98ab-063db2ecbf98 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:00.396144 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-6aaba74b-e12e-4917-98ab-063db2ecbf98 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 1096 time: 0.0794499 May 24 10:09:00.446449 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8be8c8a8-7b8e-487b-872a-12b4c8163bc8 req-8bac9e82-7c55-40d5-8658-3211ecf3b679 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:00.449488 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8be8c8a8-7b8e-487b-872a-12b4c8163bc8 req-8bac9e82-7c55-40d5-8658-3211ecf3b679 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 943 time: 0.1051440 May 24 10:09:00.482836 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-fa995c1a-4274-4270-a736-fa6186db0bc3 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8868b74e-802d-457a-bde2-4bd4e630837c HTTP/1.1" status: 200 len: 217 time: 0.0773072 May 24 10:09:00.508855 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8be8c8a8-7b8e-487b-872a-12b4c8163bc8 req-eba02e81-c547-4148-8de9-95204b04a103 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0500970 May 24 10:09:00.584819 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-49bd5ea3-6756-49c5-aa6d-1181e7075462 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:00.587321 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-49bd5ea3-6756-49c5-aa6d-1181e7075462 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0970261 May 24 10:09:00.679975 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-997cbf7a-f00a-4968-a26f-db5762d0b97e service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0836682 May 24 10:09:00.831084 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-34dc14d4-105a-4634-9d1c-f3c05832eb80 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1386490 May 24 10:09:01.000077 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bc217a65-160c-41ac-abbf-44fc53c53db0 req-0d5ea515-2785-43ac-8450-39e062b10a88 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1602387 May 24 10:09:01.768842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f04dca1f-7976-4d0b-bef1-ead0e0d9a770 req-4290a400-1de6-4cb8-be5e-0b0861e984c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:01.769729 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-f04dca1f-7976-4d0b-bef1-ead0e0d9a770 req-4290a400-1de6-4cb8-be5e-0b0861e984c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:09:01.855784 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-f04dca1f-7976-4d0b-bef1-ead0e0d9a770 req-4290a400-1de6-4cb8-be5e-0b0861e984c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:09:01.857211 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f04dca1f-7976-4d0b-bef1-ead0e0d9a770 req-4290a400-1de6-4cb8-be5e-0b0861e984c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 943 time: 0.2016158 May 24 10:09:01.907542 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f04dca1f-7976-4d0b-bef1-ead0e0d9a770 req-ddfffebf-b302-44ac-aa7f-24af1a275e5e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0437834 May 24 10:09:02.389537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 8868b74e-802d-457a-bde2-4bd4e630837c details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:09:02.535860 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=8868b74e-802d-457a-bde2-4bd4e630837c,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:02.551560 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '8868b74e-802d-457a-bde2-4bd4e630837c', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'port_id': '8868b74e-802d-457a-bde2-4bd4e630837c', 'mac_address': 'fa:16:3e:e0:38:ef', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 76, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '60925914-ac92-409e-94f9-b096645b63f1', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:09:02.625079 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:02.646639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 8868b74e-802d-457a-bde2-4bd4e630837c {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:09:02.669245 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=8868b74e-802d-457a-bde2-4bd4e630837c,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:02.838287 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:02.841219 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:02.915486 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:02.924639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.924983 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.925366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.925708 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.926070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.926391 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.926710 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.927037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.927365 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.927696 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.928065 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.928389 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:02.933973 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:03.086098 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['8868b74e-802d-457a-bde2-4bd4e630837c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:09:03.177027 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acca6216-bc61-47dc-98b0-0a30fc814482 req-2a92b290-71b3-454c-a17e-5e4df97e346f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:03.179713 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-acca6216-bc61-47dc-98b0-0a30fc814482 req-2a92b290-71b3-454c-a17e-5e4df97e346f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 944 time: 0.1455152 May 24 10:09:03.255652 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-acca6216-bc61-47dc-98b0-0a30fc814482 req-96d8e8f7-ebba-4712-890f-dae64ac389fa tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0631034 May 24 10:09:03.324066 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c08f8b00-9aaf-46bd-a99a-56330df9aea4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:04.384906 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 8868b74e-802d-457a-bde2-4bd4e630837c up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:09:04.539401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 8868b74e-802d-457a-bde2-4bd4e630837c completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:09:04.554556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 8868b74e-802d-457a-bde2-4bd4e630837c triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:09:04.555237 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:04.569950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7bca60c-e0ea-4713-8e82-7e12f7317f35 req-0f703219-6e9e-4a6a-9ceb-816c4f5e2a91 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:04.571770 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b7bca60c-e0ea-4713-8e82-7e12f7317f35 req-0f703219-6e9e-4a6a-9ceb-816c4f5e2a91 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 944 time: 0.1132760 May 24 10:09:04.652709 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b7bca60c-e0ea-4713-8e82-7e12f7317f35 req-5d9fc17b-e97a-4591-8769-470f388c36d8 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0654387 May 24 10:09:04.786999 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:04.817579 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:04.829665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=8868b74e-802d-457a-bde2-4bd4e630837c,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:04.873484 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c', 'code': 200} May 24 10:09:04.918587 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:04.921712 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:05.029741 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:05.033928 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:05.279791 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f43199b-ad9c-432f-ad27-ef5066bce187 req-e204c512-a200-411c-a270-1406f8eb6321 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:05.281327 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8f43199b-ad9c-432f-ad27-ef5066bce187 req-e204c512-a200-411c-a270-1406f8eb6321 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 1098 time: 0.0756638 May 24 10:09:05.330840 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8f43199b-ad9c-432f-ad27-ef5066bce187 req-76d2b67c-ccc2-42e1-a226-4d9491f4c519 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0428376 May 24 10:09:05.820696 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7da52696-b462-4242-8971-320f36f01ec4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:05.906697 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:05.933706 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b23d7c8-dea9-4299-88c0-97d1a873cd76 req-0fa9eef3-5ab5-46be-a0a5-c0e20c929a45 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:05.935286 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b23d7c8-dea9-4299-88c0-97d1a873cd76 req-0fa9eef3-5ab5-46be-a0a5-c0e20c929a45 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 945 time: 0.1348922 May 24 10:09:05.992259 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6b23d7c8-dea9-4299-88c0-97d1a873cd76 req-636cf3b5-ada4-46ad-ab6b-242f9e11c5bc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0466216 May 24 10:09:07.209535 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0c47675-acc7-4bc0-b292-4d2166251d6f req-3fa07305-be72-49aa-8389-4c3882bd7b66 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:07.211161 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0c47675-acc7-4bc0-b292-4d2166251d6f req-3fa07305-be72-49aa-8389-4c3882bd7b66 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 945 time: 0.0920198 May 24 10:09:07.265307 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e0c47675-acc7-4bc0-b292-4d2166251d6f req-fee52629-55af-4fed-91f2-6568b4218940 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0468569 May 24 10:09:07.283340 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:08.803594 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:11.885504 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-ffe1ef61-9703-49b6-a32c-cebef550cb98 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:11.887016 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-ffe1ef61-9703-49b6-a32c-cebef550cb98 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.1607025 May 24 10:09:11.904304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-47095c75-e407-4a01-9696-f7d43894dbfd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/quotas/8e85a9260aad4c93aa21262f8764f011 HTTP/1.1" status: 200 len: 375 time: 0.0104022 May 24 10:09:12.018181 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-82242626-5af2-4cf7-adf7-74e11a2d61dc tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1079874 May 24 10:09:12.281323 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-009e7f5f-b6b0-4434-a051-28b654b199e8 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2562349 May 24 10:09:12.429153 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-15141d0b-f253-4154-983d-5f4f00607ca5 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1396847 May 24 10:09:12.745698 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a726c279-6dd5-4296-96a7-9bce3f594dbf req-8da57e5f-1f94-4934-8d4e-815df15cfc80 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 210 time: 0.0498357 May 24 10:09:13.922024 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-79f7ff26-e16a-437f-8616-e32c120ac221 req-f396628b-b56f-4edd-aee1-315eead4d374 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 210 time: 0.0442278 May 24 10:09:15.102990 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-744474e9-c429-4dc7-8ced-c3504d7d3c59 req-c8fdc303-3c6e-44d8-a46b-9d7274c08d38 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 210 time: 0.0522234 May 24 10:09:15.598926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-45d4ab09-e16d-4008-af94-a81820a17738 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:15.601552 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-45d4ab09-e16d-4008-af94-a81820a17738 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.2259583 May 24 10:09:15.621519 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Request body: {'port': {'device_id': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'admin_state_up': True, 'tenant_id': '8e85a9260aad4c93aa21262f8764f011'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:09:15.659900 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60848) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:15.799111 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:15.820803 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:15.825628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Usage tracker for resource:security_group_rule and tenant:b7800c7d17c243cbbf14480ef7a406bd is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:15.828807 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:15.837313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:15.853960 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:15.865020 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:15.946350 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:09:15.953273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:15.964223 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.037435 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.046214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-2b97de3f-c68d-40b4-b383-80f09d623e8a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:16.090776 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=b7800c7d17c243cbbf14480ef7a406bd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4296672 May 24 10:09:16.091880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=507f9bc0-6a00-4625-b244-5fc0857f30fe,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:16.095054 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60858) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:16.118826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.119972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.119972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.119972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.120494 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.120666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.120994 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.127363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.127363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.127363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.127363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.127363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:09:16.131252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9b743d40-875a-431e-9132-550c39591fa6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:16.149188 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Allocated IP 172.20.0.7 (52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd/60925914-ac92-409e-94f9-b096645b63f1/aeb4b23e-f182-4da2-a23f-751d3b29c315) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:09:16.182650 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.216869 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.397827 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c4ce595e-6e00-43bc-b713-b4db138400d0 req-308cb210-5731-4499-a8f8-de836abab1ba tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 210 time: 0.0633221 May 24 10:09:16.398173 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.403062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d0158fb4-1f2e-4007-b989-1fd93e2c1f06 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:16.419660 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=507f9bc0-6a00-4625-b244-5fc0857f30fe,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:16.422376 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:16.441724 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.443530 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/507f9bc0-6a00-4625-b244-5fc0857f30fe HTTP/1.1" status: 204 len: 168 time: 0.3468165 May 24 10:09:16.458466 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.563767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.570542 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-36a2d848-6f3e-4486-81a7-3fe256a6f7e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:16.584891 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d', '8868b74e-802d-457a-bde2-4bd4e630837c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:09:16.643438 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:09:16.646905 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:09:16.651825 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:09:16.694199 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event updated for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:16.702486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:16.753405 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:16.754615 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:16.918779 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3049936 May 24 10:09:16.936424 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Request body: {'port': {'device_id': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:09:17.094120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.307738 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60868) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:17.327442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.344833 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:17.349482 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Usage tracker for resource:security_group_rule and tenant:6601ff8e69be46c98a72751830ae013f is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:17.352720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:17.359812 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:17.461463 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.528476 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id aeb4b23e-f182-4da2-a23f-751d3b29c315 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:09:17.536602 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.552648 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Transition to ACTIVE for port object aeb4b23e-f182-4da2-a23f-751d3b29c315 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:09:17.565115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.576805 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5956548d-fe5f-4e59-a924-b163ae03e89f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:17.626251 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.630814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-63fe180d-daba-4609-a7df-a8e768069f91 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:17.674601 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=94a3260f-b109-4608-a4f8-dadbb9febdc0,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:17.682647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:17.700834 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6601ff8e69be46c98a72751830ae013f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3916416 May 24 10:09:17.704442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60874) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:17.785545 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb1fa9f2-2f1a-4f72-91f1-c01c7144d624 req-cd7c3485-e55f-4ff4-8e81-94f2945e2cb7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:17.787437 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb1fa9f2-2f1a-4f72-91f1-c01c7144d624 req-cd7c3485-e55f-4ff4-8e81-94f2945e2cb7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 943 time: 0.1761129 May 24 10:09:17.798263 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.887724 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cb1fa9f2-2f1a-4f72-91f1-c01c7144d624 req-3ea96921-0827-46e8-8b23-1f0983951de0 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0829194 May 24 10:09:17.892704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:17.990198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Pushing event updated for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:18.017911 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.387s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:18.083834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:18.091438 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:18.113469 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=94a3260f-b109-4608-a4f8-dadbb9febdc0,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:18.118693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:18.120170 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Attempting to bind port aeb4b23e-f182-4da2-a23f-751d3b29c315 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:09:18.120842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Attempting to bind port aeb4b23e-f182-4da2-a23f-751d3b29c315 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:09:18.121267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Attempting to bind port aeb4b23e-f182-4da2-a23f-751d3b29c315 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:09:18.122683 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:18.128642 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c25b24b8-f5ed-4679-990d-216ca6ee0125 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/94a3260f-b109-4608-a4f8-dadbb9febdc0 HTTP/1.1" status: 204 len: 168 time: 0.4186547 May 24 10:09:18.129455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Port aeb4b23e-f182-4da2-a23f-751d3b29c315 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:09:18.129956 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Attempting to bind port aeb4b23e-f182-4da2-a23f-751d3b29c315 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:09:18.140284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 8, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:09:18.140822 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Checking segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:09:18.141455 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Bound using segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:09:18.142177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Bound port: aeb4b23e-f182-4da2-a23f-751d3b29c315, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:09:18.142749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:18.294634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:09:18.300422 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:09:18.386861 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:18.391178 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-34b53ba6-55a1-4876-92f4-f62b16da1010 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:18.461489 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:18.462227 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:18.463686 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] 10.222.0.95 "PUT /v2.0/ports/aeb4b23e-f182-4da2-a23f-751d3b29c315 HTTP/1.1" status: 200 len: 1093 time: 1.5364635 May 24 10:09:18.466556 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:18.493540 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Pushing event updated for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:18.497582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:18.516820 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'status': 'completed', 'code': 200} May 24 10:09:18.633935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-445ab60c-5064-4bb7-909a-340d8390db2c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:18.642433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-445ab60c-5064-4bb7-909a-340d8390db2c service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 1096 time: 0.0836291 May 24 10:09:18.864841 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-449daa27-b761-495b-9115-31def2eb7efa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:18.866494 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-449daa27-b761-495b-9115-31def2eb7efa service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=8e85a9260aad4c93aa21262f8764f011&shared=False HTTP/1.1" status: 200 len: 915 time: 0.2216430 May 24 10:09:19.080622 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-1ded42fb-65ef-46a5-a36f-b34b7fdc8826 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:19.082898 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7d05f146-89b6-4d29-b001-66d4ab7c545b req-1ded42fb-65ef-46a5-a36f-b34b7fdc8826 service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.2080355 May 24 10:09:19.124335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4db96e10-6e42-4c2f-80c1-ad116bfe5ab5 req-23c102c4-b005-4e63-b146-1c1cf26d764f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:19.127200 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4db96e10-6e42-4c2f-80c1-ad116bfe5ab5 req-23c102c4-b005-4e63-b146-1c1cf26d764f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 943 time: 0.1079476 May 24 10:09:19.189893 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4db96e10-6e42-4c2f-80c1-ad116bfe5ab5 req-008e8985-be0c-4dbe-9990-ac1f435a5074 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0563483 May 24 10:09:19.218058 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-5fe68fd7-9cf7-492f-81e0-8d667d82b4e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:19.219714 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-5fe68fd7-9cf7-492f-81e0-8d667d82b4e5 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 1096 time: 0.0921249 May 24 10:09:19.312077 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-b45ec41b-2418-4a2c-a6b5-5ded7f47213f service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315 HTTP/1.1" status: 200 len: 217 time: 0.0768743 May 24 10:09:19.379433 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-c85fa335-4d69-4d3a-a621-ef34502264d6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:19.381093 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-c85fa335-4d69-4d3a-a621-ef34502264d6 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0666480 May 24 10:09:19.434684 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-bf5a817c-87a4-42c6-8ccd-91ef8d1b08b5 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0443099 May 24 10:09:19.597165 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-6b4807e6-37b7-4ce9-9b4c-80ff57442d4f service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1521626 May 24 10:09:19.753918 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8e663f0b-8d44-4425-8bcc-22b1e094ff26 req-916019c5-c5e1-46d7-ba37-b48916a25338 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1447413 May 24 10:09:20.215152 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:20.409277 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d958cfc-be41-4f6c-aaeb-44f7f42f07a3 req-ff7283c8-98e5-4075-af42-171e4eccb2d7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:20.410025 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3d958cfc-be41-4f6c-aaeb-44f7f42f07a3 req-ff7283c8-98e5-4075-af42-171e4eccb2d7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:09:20.511663 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3d958cfc-be41-4f6c-aaeb-44f7f42f07a3 req-ff7283c8-98e5-4075-af42-171e4eccb2d7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:09:20.512841 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3d958cfc-be41-4f6c-aaeb-44f7f42f07a3 req-ff7283c8-98e5-4075-af42-171e4eccb2d7 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 943 time: 0.1985598 May 24 10:09:20.573100 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3d958cfc-be41-4f6c-aaeb-44f7f42f07a3 req-d7dcc9a3-56b3-4c0b-914c-89af39913704 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0530081 May 24 10:09:21.799802 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd6dd3ae-40bc-48bb-9b92-913cb65706fe req-c26338c3-bf39-45f3-a901-0d3772b13ab3 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:21.804214 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bd6dd3ae-40bc-48bb-9b92-913cb65706fe req-c26338c3-bf39-45f3-a901-0d3772b13ab3 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 943 time: 0.1084793 May 24 10:09:21.852802 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bd6dd3ae-40bc-48bb-9b92-913cb65706fe req-0fe9d0f5-6c93-43de-b9e5-8974b14d12d8 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444708 May 24 10:09:22.484949 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device aeb4b23e-f182-4da2-a23f-751d3b29c315 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:09:22.490474 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:09:22.669039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:22.684720 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'port_id': 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'mac_address': 'fa:16:3e:18:54:04', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 76, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '60925914-ac92-409e-94f9-b096645b63f1', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:09:22.747048 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:22.765594 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id aeb4b23e-f182-4da2-a23f-751d3b29c315 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:09:22.782848 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:22.871913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:22.875302 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:22.953337 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:22.956826 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:23.135907 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec580ef-c433-4038-9753-638d7d4ca52e req-5e251c2d-9c8a-4aab-aff8-92bbfa61c00e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:23.137662 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ec580ef-c433-4038-9753-638d7d4ca52e req-5e251c2d-9c8a-4aab-aff8-92bbfa61c00e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 944 time: 0.1236942 May 24 10:09:23.157937 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['aeb4b23e-f182-4da2-a23f-751d3b29c315'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:09:23.190036 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0ec580ef-c433-4038-9753-638d7d4ca52e req-99ee8498-df79-448f-965f-b3c19d52a4ea tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0449917 May 24 10:09:24.399363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85eda884-c8d9-4e2d-ba11-4273c58a601d req-28aa6abf-01c5-47f9-a087-22d0253ca219 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:24.401451 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85eda884-c8d9-4e2d-ba11-4273c58a601d req-28aa6abf-01c5-47f9-a087-22d0253ca219 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 944 time: 0.0900631 May 24 10:09:24.451799 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-85eda884-c8d9-4e2d-ba11-4273c58a601d req-c0513ce5-ff3f-4561-b274-e18be9a11727 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441051 May 24 10:09:24.513291 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device aeb4b23e-f182-4da2-a23f-751d3b29c315 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:09:24.683911 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port aeb4b23e-f182-4da2-a23f-751d3b29c315 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:09:24.691416 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port aeb4b23e-f182-4da2-a23f-751d3b29c315 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:09:24.692064 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:24.896308 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:24.951077 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:24.974411 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:25.003318 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'code': 200} May 24 10:09:25.060657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:25.063850 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-d293e433-d98b-4f70-ad9b-b1917def0612 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:25.136567 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:25.140439 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:25.318686 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60962) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:25.412938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e251306-433b-4f9c-81f9-c086c7fd63a7 req-887b9853-06b4-4475-8f2d-50ac080450eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:25.414898 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e251306-433b-4f9c-81f9-c086c7fd63a7 req-887b9853-06b4-4475-8f2d-50ac080450eb service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 1098 time: 0.0779097 May 24 10:09:25.465570 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e251306-433b-4f9c-81f9-c086c7fd63a7 req-d76eb296-86f7-46b6-8530-c91640b10bdb service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436990 May 24 10:09:25.475576 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:25.495834 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:25.502385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Usage tracker for resource:security_group_rule and tenant:17c45c335cd64e1fa50e7f497fe9a2d0 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:25.507781 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:25.518929 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:25.690026 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:25.703841 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-280d1b4a-aede-4374-9149-6061ca124c17 req-f6b4940a-1ede-46c3-891e-f02595f5da6f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:25.705684 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-280d1b4a-aede-4374-9149-6061ca124c17 req-f6b4940a-1ede-46c3-891e-f02595f5da6f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 945 time: 0.1396747 May 24 10:09:25.764387 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-280d1b4a-aede-4374-9149-6061ca124c17 req-af822b72-0795-491b-9cc5-cdf204af1d16 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0491090 May 24 10:09:25.774856 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:25.788963 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e5fc6c53-661b-47c9-ba0b-45c182f8a0ff admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:25.937555 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=17c45c335cd64e1fa50e7f497fe9a2d0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6172838 May 24 10:09:25.939680 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8674ff71-ad35-4832-9404-6684a0eed23d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:25.940803 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60970) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:25.943538 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:26.049160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:26.096171 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:26.291940 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:26.296178 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9d8a2eda-6056-4b65-adbe-14806e8b60d7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:26.312546 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8674ff71-ad35-4832-9404-6684a0eed23d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:26.317611 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:26.323019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:26.324612 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-98eea93d-76f0-47e0-ac43-2fb10411d88f admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/8674ff71-ad35-4832-9404-6684a0eed23d HTTP/1.1" status: 204 len: 168 time: 0.3824196 May 24 10:09:27.140463 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ecf920f0-6db2-4ca5-812f-0033e021e286 req-80d16e77-eeaa-4dc5-b27a-e9c037692f99 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:27.145598 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ecf920f0-6db2-4ca5-812f-0033e021e286 req-80d16e77-eeaa-4dc5-b27a-e9c037692f99 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 945 time: 0.2238538 May 24 10:09:27.203846 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ecf920f0-6db2-4ca5-812f-0033e021e286 req-982528f0-bad5-4657-94de-43a55fb672be tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0523984 May 24 10:09:27.214309 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 60980) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:27.235333 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.252563 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:27.258205 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Usage tracker for resource:security_group_rule and tenant:bc5bf92e344647f5b18aa6b88eb690ef is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:27.262737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:27.270663 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:27.389572 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.466175 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.481767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b7254b80-ed1c-4cec-91b6-f79e7ba0ac63 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:27.536563 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=bc5bf92e344647f5b18aa6b88eb690ef&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3206346 May 24 10:09:27.541346 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=37f3d25a-9900-4db0-a07f-502c2afcb019,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:27.541549 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 60986) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:27.552147 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-035f4dc9-2237-4cdb-b33a-292927805308 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:27.636505 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.667701 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.814819 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.823313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:27.838689 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=37f3d25a-9900-4db0-a07f-502c2afcb019,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:27.841984 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:27.846756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:27.847895 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-72ecb22f-5bb1-4ca9-bc24-b09556f3a020 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/37f3d25a-9900-4db0-a07f-502c2afcb019 HTTP/1.1" status: 204 len: 168 time: 0.3055534 May 24 10:09:33.198193 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d040e6f0-36f1-488d-a6e9-a0f63af3e35b req-d4d1c9ef-576c-4d5c-80c3-adf51f6041e0 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:33.200431 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d040e6f0-36f1-488d-a6e9-a0f63af3e35b req-d4d1c9ef-576c-4d5c-80c3-adf51f6041e0 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1638699 May 24 10:09:33.263269 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d040e6f0-36f1-488d-a6e9-a0f63af3e35b req-04d5731a-f645-4feb-9f5c-823287676d4c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0565386 May 24 10:09:33.324210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d557c6b9-0403-452c-a5a2-f21401f4522e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:33.352735 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 32850) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:33.503840 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:33.520659 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:33.525208 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Usage tracker for resource:security_group_rule and tenant:706537a664784420bd5e4cf9b3806a80 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:33.528595 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:33.535656 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:33.654030 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:33.751601 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:33.761476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5ea884af-319b-4824-a55c-45e59006155f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:33.808882 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=706537a664784420bd5e4cf9b3806a80&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4543383 May 24 10:09:33.810810 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9caf5b16-60e8-4b45-a7d0-69b4b4d0bda7,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:33.812958 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 32856) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:33.814136 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:33.904231 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:33.933638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:34.111567 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:34.115605 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e5f7f3f4-14d8-4df4-b8b0-88c315aea53f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:34.130503 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9caf5b16-60e8-4b45-a7d0-69b4b4d0bda7,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:34.134547 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:34.141914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:34.143297 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-42205eea-a86c-4ddd-9bf3-e51144034cf4 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/9caf5b16-60e8-4b45-a7d0-69b4b4d0bda7 HTTP/1.1" status: 204 len: 168 time: 0.3288112 May 24 10:09:34.641377 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59004045-9a9e-47be-a6ac-2d84b175d86d req-a8dfd2aa-36ad-4f8d-9231-b0d4b9f2e0d3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:34.642893 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-59004045-9a9e-47be-a6ac-2d84b175d86d req-a8dfd2aa-36ad-4f8d-9231-b0d4b9f2e0d3 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1344128 May 24 10:09:34.703883 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-59004045-9a9e-47be-a6ac-2d84b175d86d req-87757365-f7af-4c51-84ea-48832679421e tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0505919 May 24 10:09:34.915668 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 32868) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:34.934628 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:34.952012 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:34.956393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Usage tracker for resource:security_group_rule and tenant:83a3e0bbc05f4f6d9106ed8bd71d0b5c is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:34.959787 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:34.967664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:35.064324 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.152178 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.168248 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2b25482f-1e46-424e-9c99-fa0d0886ab73 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:35.244394 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=83a3e0bbc05f4f6d9106ed8bd71d0b5c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3265984 May 24 10:09:35.248437 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1c418f94-952a-4db2-b4a8-525a3216a4a1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:35.248590 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 32872) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:35.250941 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-09f56f4c-cae7-4be0-8607-dba3d99ce6d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:35.318183 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.349342 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.546253 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.554927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-bd3446df-4dd6-4469-a8a7-4d1663b4f159 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:35.581410 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1c418f94-952a-4db2-b4a8-525a3216a4a1,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:35.585603 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:35.595862 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.597003 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-713aac68-c9ba-4945-96ad-44becc005b99 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/1c418f94-952a-4db2-b4a8-525a3216a4a1 HTTP/1.1" status: 204 len: 168 time: 0.3489528 May 24 10:09:35.854783 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ca5cb28d-b1e2-4db8-a160-2940a45205e2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.890120 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:35.975279 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-417b2e7d-3ab8-4c23-a119-81a41bb2ac87 req-c128f3e7-d8a6-482d-92c5-825a78ed6163 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:35.977196 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-417b2e7d-3ab8-4c23-a119-81a41bb2ac87 req-c128f3e7-d8a6-482d-92c5-825a78ed6163 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0973303 May 24 10:09:36.044129 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-417b2e7d-3ab8-4c23-a119-81a41bb2ac87 req-ea01e8c8-d331-4d2b-8aad-36c7a125e38d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0592031 May 24 10:09:37.208711 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:37.311013 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60be9b48-7426-48a5-bf5a-e7f0470fec45 req-b6dab98e-b9fe-4c46-a2e1-66a766398e49 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:37.312628 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-60be9b48-7426-48a5-bf5a-e7f0470fec45 req-b6dab98e-b9fe-4c46-a2e1-66a766398e49 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1468308 May 24 10:09:37.368995 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-60be9b48-7426-48a5-bf5a-e7f0470fec45 req-f76d52e6-9705-477e-9f6b-0a7f4983b99e tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0495808 May 24 10:09:38.576413 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d49d2934-28b9-42db-bb50-5eb823fe93b5 req-264f3e05-ccb2-46a7-9cd6-7ecc0b0f65c8 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:38.578157 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d49d2934-28b9-42db-bb50-5eb823fe93b5 req-264f3e05-ccb2-46a7-9cd6-7ecc0b0f65c8 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0922377 May 24 10:09:38.630603 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d49d2934-28b9-42db-bb50-5eb823fe93b5 req-ef625981-fcfb-42b6-acd3-435368db4481 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0448098 May 24 10:09:38.768727 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:39.660341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 32958) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:39.860172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:39.881165 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:39.882928 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a92c5ab-8548-4ed8-85ac-5391c5d66ab6 req-00214014-fa44-46a7-8cd2-7f0d15356b68 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:39.886197 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a92c5ab-8548-4ed8-85ac-5391c5d66ab6 req-00214014-fa44-46a7-8cd2-7f0d15356b68 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1234963 May 24 10:09:39.888610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Usage tracker for resource:security_group_rule and tenant:377e0106bd2245e89fb610a6e7d9f309 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:39.894433 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:39.903171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:39.973682 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-7a92c5ab-8548-4ed8-85ac-5391c5d66ab6 req-92cb5668-769e-4a53-9aed-32a239a5a87e tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0751457 May 24 10:09:40.053958 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.135201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.151489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-894db1bb-c575-422d-818c-2c50a78273ad admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:40.193900 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=377e0106bd2245e89fb610a6e7d9f309&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5320876 May 24 10:09:40.195783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1f187d5e-060d-4495-b399-e164d53a647a,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:40.197656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 32968) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:40.199641 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:40.271100 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.302062 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.459926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.464425 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a2666aa7-18bd-4520-821c-dd9c35a72c52 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:40.478846 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1f187d5e-060d-4495-b399-e164d53a647a,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:40.482871 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:40.489770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:40.490907 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/1f187d5e-060d-4495-b399-e164d53a647a HTTP/1.1" status: 204 len: 168 time: 0.2915208 May 24 10:09:41.271075 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d16dac0a-2014-4b64-9571-dac6df65c188 req-af4673c6-4786-43ed-bc25-bfeee69fbb5d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:41.272705 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d16dac0a-2014-4b64-9571-dac6df65c188 req-af4673c6-4786-43ed-bc25-bfeee69fbb5d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0996368 May 24 10:09:41.329186 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d16dac0a-2014-4b64-9571-dac6df65c188 req-27748771-e62f-4472-a551-31abdd9930f1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0478156 May 24 10:09:41.357335 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 32980) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:41.377242 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.394637 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:41.399583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Usage tracker for resource:security_group_rule and tenant:5349a640536e4850976b685359eae207 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:41.403537 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:41.410714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:41.505803 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.591296 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.599108 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-225466f4-7f33-42ec-b979-bd2d5cfdbff5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:41.648552 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=5349a640536e4850976b685359eae207&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2900331 May 24 10:09:41.649714 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b33caca5-a575-4e05-ac95-b13c98567dbd,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:41.652388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 32984) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:41.653800 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-6222eb45-ddfa-4ffb-a308-f35bc1c9c0e2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:41.721478 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.751039 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.911832 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.916014 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-92e5ff8c-7439-4dec-9033-7266952dbada admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:41.930133 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b33caca5-a575-4e05-ac95-b13c98567dbd,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:41.934653 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:41.940966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:41.942379 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-cc986333-e68b-4007-88bd-b9bf436901da admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b33caca5-a575-4e05-ac95-b13c98567dbd HTTP/1.1" status: 204 len: 168 time: 0.2883584 May 24 10:09:42.595364 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d143948d-ae5c-4ac8-b45d-d82bd5579997 req-e627fa79-17fb-47c2-8bc3-2f262977918c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:42.597023 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d143948d-ae5c-4ac8-b45d-d82bd5579997 req-e627fa79-17fb-47c2-8bc3-2f262977918c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1448565 May 24 10:09:42.650604 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d143948d-ae5c-4ac8-b45d-d82bd5579997 req-ef7f9a87-ec55-433c-ae2b-764f227dbb16 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0464101 May 24 10:09:42.753692 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9e84f44-7591-4d51-bcf6-1920e0b4393c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:42.754636 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9e84f44-7591-4d51-bcf6-1920e0b4393c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:09:42.842860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9e84f44-7591-4d51-bcf6-1920e0b4393c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:09:42.844262 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9e84f44-7591-4d51-bcf6-1920e0b4393c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.3385730 May 24 10:09:42.861036 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-206a2766-e183-4b04-96e2-85480044b547 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/quotas/8e85a9260aad4c93aa21262f8764f011 HTTP/1.1" status: 200 len: 375 time: 0.0105393 May 24 10:09:42.966407 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-5e402c10-7012-495e-b241-cea6f737f4dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0977640 May 24 10:09:43.124781 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-5c72291e-a7b7-4fd6-b551-02566f460492 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1454942 May 24 10:09:43.285249 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-1e87aa06-5ab4-4f22-8108-211caec41db7 service neutron] 10.222.0.67 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1530695 May 24 10:09:43.650297 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-bb5956fb-5d1a-4216-a5ea-23a772078ba1 req-f5cbbce5-129e-4aea-86af-80ea579808ca tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 210 time: 0.0481455 May 24 10:09:43.866534 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f361ea8-fd54-401f-bcbd-dc5073684e33 req-ce09b1b3-0161-4643-888a-d93bb5f1588a tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:43.868053 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f361ea8-fd54-401f-bcbd-dc5073684e33 req-ce09b1b3-0161-4643-888a-d93bb5f1588a tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0928864 May 24 10:09:43.925690 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f361ea8-fd54-401f-bcbd-dc5073684e33 req-da953801-4036-427c-9180-61fb106a641d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0505514 May 24 10:09:44.080744 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33042) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:44.218511 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.246163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:44.252669 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Usage tracker for resource:security_group_rule and tenant:3dfac74456894bbdbff58d0a955fc51f is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:09:44.257641 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:09:44.268260 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:44.365744 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.442740 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.451667 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-841d0b16-691d-43d1-93de-48a02213ca49 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:44.501011 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=3dfac74456894bbdbff58d0a955fc51f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4189513 May 24 10:09:44.503160 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9386068b-c1f7-446b-851c-17140e9287b0,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:44.504505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33050) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:09:44.508505 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6efa73dd-d8e7-4e26-886e-0e04dfcf1b30 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:44.588806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.628926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.821656 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.830295 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-48cec3c0-80f1-4605-83c8-2db01086b277 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:44.840382 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-dd67f613-e2f8-4be1-9cd0-91183c84ed66 req-282b73e8-2d9e-4f65-a696-55d4f3826df3 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 210 time: 0.0571148 May 24 10:09:44.847686 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9386068b-c1f7-446b-851c-17140e9287b0,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:44.851963 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:44.859198 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:44.861584 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-708341a9-41ca-49a4-881c-74c7d40ac2b2 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/9386068b-c1f7-446b-851c-17140e9287b0 HTTP/1.1" status: 204 len: 168 time: 0.3556495 May 24 10:09:45.129915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef5268ae-887b-41e9-996d-e0a0d066b551 req-187f7a00-644d-4eb8-8f5a-faf2242ccc09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:45.131389 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ef5268ae-887b-41e9-996d-e0a0d066b551 req-187f7a00-644d-4eb8-8f5a-faf2242ccc09 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0896034 May 24 10:09:45.189238 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ef5268ae-887b-41e9-996d-e0a0d066b551 req-b23172b4-c5b9-42aa-87fa-b6e324db4b43 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0499396 May 24 10:09:46.013223 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-06ff5f43-36ff-4f78-b94d-6c4e40ac74d8 req-4a4c2f6c-3ffb-44a0-adbd-6d2a075cbf72 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 210 time: 0.0503385 May 24 10:09:46.387824 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-5e0b1545-1939-42b1-a56a-19894cfcf4a2 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:46.390323 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-5e0b1545-1939-42b1-a56a-19894cfcf4a2 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 815 time: 0.2042487 May 24 10:09:46.408901 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Request body: {'port': {'device_id': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'admin_state_up': True, 'tenant_id': '8e85a9260aad4c93aa21262f8764f011'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:09:46.488100 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff0d14ee-cab3-4426-b077-5c3438eb2dfd req-0de62226-201f-4172-8e6a-c5065b946efa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:46.490852 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ff0d14ee-cab3-4426-b077-5c3438eb2dfd req-0de62226-201f-4172-8e6a-c5065b946efa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1658249 May 24 10:09:46.586669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:09:46.601999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:09:46.668710 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ff0d14ee-cab3-4426-b077-5c3438eb2dfd req-81a5c383-7edc-4240-b0e5-63543a29cb1d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1656005 May 24 10:09:46.729271 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Made reservation on behalf of 8e85a9260aad4c93aa21262f8764f011 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:09:46.733558 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:46.933634 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Allocated IP 172.20.0.13 (52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd/60925914-ac92-409e-94f9-b096645b63f1/f24b7dd1-0def-4abf-89e7-2699ed5e0fe1) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:09:47.336965 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d6b04acf-ae98-4380-9479-84548d376f3b req-ee519e40-6239-4390-a51e-2261f2c568f0 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 210 time: 0.0590546 May 24 10:09:47.388808 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:47.511892 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:47.517914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4615b39-ddde-4da6-b4a5-ccca54c7dcb9 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:47.523546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d', '8868b74e-802d-457a-bde2-4bd4e630837c', 'aeb4b23e-f182-4da2-a23f-751d3b29c315'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:09:47.586842 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:09:47.591325 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:09:47.595701 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:09:47.636760 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event updated for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:47.648475 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:47.673484 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:47.674500 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:47.804860 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.4000940 May 24 10:09:47.820807 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Request body: {'port': {'device_id': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:09:47.961276 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82210cd4-2622-494f-9f57-4423c614ca97 req-c9a7b081-305d-4be5-9ba8-6b4c043d6fc1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:47.969567 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82210cd4-2622-494f-9f57-4423c614ca97 req-c9a7b081-305d-4be5-9ba8-6b4c043d6fc1 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1595883 May 24 10:09:47.971941 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:48.050112 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-82210cd4-2622-494f-9f57-4423c614ca97 req-d54aaded-da44-451f-9b32-011d8cd276f6 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0741222 May 24 10:09:48.359688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:09:48.373287 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:48.401747 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Transition to ACTIVE for port object f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:09:48.454462 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:48.463336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:48.515222 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1e19eff-2321-4a16-858a-b884163879bc req-570af6e2-78b0-40e3-9a57-0adcd2de529a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:48.517806 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-570af6e2-78b0-40e3-9a57-0adcd2de529a service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.1095848 May 24 10:09:48.707003 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Pushing event updated for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:48.717580 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.255s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:48.744068 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1e19eff-2321-4a16-858a-b884163879bc req-7bcb3076-9e80-49d1-a6bd-1008a35d6fc8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:48.746053 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-7bcb3076-9e80-49d1-a6bd-1008a35d6fc8 service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.2210867 May 24 10:09:48.824271 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65d0f513-bbe0-4206-9a21-fc0a15aa3fe4 req-1533d13f-9757-43a1-a3b3-060f2a100e76 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:48.825034 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-65d0f513-bbe0-4206-9a21-fc0a15aa3fe4 req-1533d13f-9757-43a1-a3b3-060f2a100e76 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:09:48.830578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Attempting to bind port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:09:48.830910 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Attempting to bind port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:09:48.831238 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Attempting to bind port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:09:48.836914 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:09:48.837956 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Attempting to bind port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 on network 52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:09:48.849107 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 9, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:09:48.849458 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Checking segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:09:48.849839 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Bound using segment: {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:09:48.850284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Bound port: f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b42a154a-6873-4e20-9753-b2afc6ed11b0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 76, 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:09:48.850637 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:48.927231 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-65d0f513-bbe0-4206-9a21-fc0a15aa3fe4 req-1533d13f-9757-43a1-a3b3-060f2a100e76 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:09:48.929044 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-65d0f513-bbe0-4206-9a21-fc0a15aa3fe4 req-1533d13f-9757-43a1-a3b3-060f2a100e76 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 944 time: 0.4080296 May 24 10:09:48.939179 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-cf0fbd32-2db8-4059-9ba7-ff3545f53410 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 1000 time: 0.1832445 May 24 10:09:49.010520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1e19eff-2321-4a16-858a-b884163879bc req-e40f8f93-961b-4074-b539-a5dc5451fa85 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.012198 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-e40f8f93-961b-4074-b539-a5dc5451fa85 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.0656643 May 24 10:09:49.035972 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:09:49.052208 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-65d0f513-bbe0-4206-9a21-fc0a15aa3fe4 req-54c73278-5871-496a-a47a-3f6897e14a3f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1123011 May 24 10:09:49.056284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:09:49.104960 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1e19eff-2321-4a16-858a-b884163879bc req-922db6bc-aeb5-4416-b63f-c52d7823def4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.106537 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-922db6bc-aeb5-4416-b63f-c52d7823def4 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0865667 May 24 10:09:49.130284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:49.139980 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-95db9554-a4d0-490b-8903-54fecfcc0988 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:49.247369 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.248395 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:49.251027 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] 10.222.0.95 "PUT /v2.0/ports/f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 HTTP/1.1" status: 200 len: 1094 time: 1.4362235 May 24 10:09:49.257811 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:49.302539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Pushing event updated for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:49.309658 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:49.313646 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-5b28215b-6d28-4a2d-9a03-b06aa244be12 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1981108 May 24 10:09:49.347077 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1', 'status': 'completed', 'code': 200} May 24 10:09:49.491305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-307ce5bd-6d1c-45e7-83d0-32130ef282d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.493836 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-307ce5bd-6d1c-45e7-83d0-32130ef282d8 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 1097 time: 0.1309648 May 24 10:09:49.496561 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-12f8842d-9d13-4480-ac93-1c01704e750f req-84ea5529-e9cc-430b-be3f-05fef534d927 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.498333 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-12f8842d-9d13-4480-ac93-1c01704e750f req-84ea5529-e9cc-430b-be3f-05fef534d927 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.2694054 May 24 10:09:49.623734 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-545024e5-28ed-4cb9-af58-326739b5d119 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 HTTP/1.1" status: 200 len: 217 time: 0.1205444 May 24 10:09:49.625412 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-12f8842d-9d13-4480-ac93-1c01704e750f req-007d70ae-daa5-43b8-b0e2-ce3eccdaf023 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1111810 May 24 10:09:49.627677 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-a1b58eba-8159-4825-b36a-6c8dc40025a3 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3047752 May 24 10:09:49.706919 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9b24075-5f23-46b6-821e-3ec1387381e4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:49.708402 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-e9b24075-5f23-46b6-821e-3ec1387381e4 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0711615 May 24 10:09:49.762998 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-b0dc8369-83d2-44ca-a9ea-1451a86b6825 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0468826 May 24 10:09:50.039874 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-24990fa9-a71b-4d1d-8409-cc735bf9bc00 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2688448 May 24 10:09:50.192824 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:50.221765 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ae7ecaf9-338f-4772-b9a0-1e2b1a3bc78f req-89b5a63e-db27-407d-9cad-db8435d56029 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1707263 May 24 10:09:50.318516 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0316f04c-9272-40d8-b831-e879df344fc5 req-814eda3c-8afc-4d6f-93e4-977482d25801 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:50.320040 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0316f04c-9272-40d8-b831-e879df344fc5 req-814eda3c-8afc-4d6f-93e4-977482d25801 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 944 time: 0.0972357 May 24 10:09:50.435967 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0316f04c-9272-40d8-b831-e879df344fc5 req-af6014b5-a46a-4f2f-a3dd-0c35eb255a21 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1019926 May 24 10:09:50.465009 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-b8814d5a-7dc1-4f41-a8e4-49e2631419e9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:50.467463 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-b8814d5a-7dc1-4f41-a8e4-49e2631419e9 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 1097 time: 0.1024380 May 24 10:09:50.717915 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-67bba7e5-8eea-4d4e-baa0-c11c3d2da577 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:50.720127 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-67bba7e5-8eea-4d4e-baa0-c11c3d2da577 service neutron] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 915 time: 0.2457764 May 24 10:09:50.832627 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-814e4d38-af3d-47c5-aa8f-b72fdf59cbe9 req-cfe60379-4bf4-48d5-a128-75424273e796 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:50.834368 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-814e4d38-af3d-47c5-aa8f-b72fdf59cbe9 req-cfe60379-4bf4-48d5-a128-75424273e796 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0913012 May 24 10:09:50.853304 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-2b9cbb8b-51bb-4fb8-af08-5c9d015e05e7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 HTTP/1.1" status: 200 len: 217 time: 0.1157696 May 24 10:09:50.902761 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-814e4d38-af3d-47c5-aa8f-b72fdf59cbe9 req-65e2bffb-6f51-4746-84e1-c61b41527f6e tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0602679 May 24 10:09:50.932670 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-384abaae-1d56-4bde-a0c9-f9e71c617700 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:50.934240 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-384abaae-1d56-4bde-a0c9-f9e71c617700 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0681825 May 24 10:09:50.989071 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-27133585-beb2-4b31-a2a2-67d405de47c2 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0444539 May 24 10:09:51.136509 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-0e6b0148-3cec-44b7-80c6-c1204a7c9dc2 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1370935 May 24 10:09:51.280087 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e5f443be-7e4f-44b8-90db-1ac14967423f req-22b322be-5dc6-4b22-a94f-6489e24548bd service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1341064 May 24 10:09:51.514547 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-7eba78fe-925b-4740-b92f-e4c20c844502 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0932136 May 24 10:09:51.694369 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-1bf40458-e0b8-4fc3-841e-1860bcf29315 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:51.696143 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-1bf40458-e0b8-4fc3-841e-1860bcf29315 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=8e85a9260aad4c93aa21262f8764f011&device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 1098 time: 0.1545494 May 24 10:09:51.711027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab1e347b-96e8-4da9-8abd-8ef5932b9b91 req-1a51e032-66c4-4e66-8497-3f61fc592776 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:51.713257 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ab1e347b-96e8-4da9-8abd-8ef5932b9b91 req-1a51e032-66c4-4e66-8497-3f61fc592776 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 944 time: 0.1140375 May 24 10:09:51.865578 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ab1e347b-96e8-4da9-8abd-8ef5932b9b91 req-91b7ffc4-4ab6-439d-87e0-60f0b847153c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1452858 May 24 10:09:51.937235 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-c5d8402c-ba79-4945-b1e4-afc5d208620e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:51.938813 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-c5d8402c-ba79-4945-b1e4-afc5d208620e service neutron] 10.222.0.95 "GET /v2.0/networks?id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 200 len: 915 time: 0.2034047 May 24 10:09:52.013167 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-2fb9791f-de2c-45b0-94e9-dfa5d7c58add service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=3768267d-8ff1-4093-bb55-85e4456cd30d HTTP/1.1" status: 200 len: 217 time: 0.0668449 May 24 10:09:52.086800 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-86215ae1-e91c-43f9-a4cb-14d1d0293dd1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:52.089122 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-86215ae1-e91c-43f9-a4cb-14d1d0293dd1 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 200 len: 855 time: 0.0693836 May 24 10:09:52.140630 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-7a28b816-1728-45f7-9601-7cbc4db5e14e service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0443146 May 24 10:09:52.196546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f110800-9a4d-4231-ad32-a7c4ceca5632 req-2bc72ebe-7f66-4a88-b3bd-6236ba7a5688 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:52.199815 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f110800-9a4d-4231-ad32-a7c4ceca5632 req-2bc72ebe-7f66-4a88-b3bd-6236ba7a5688 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1077316 May 24 10:09:52.257509 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8f110800-9a4d-4231-ad32-a7c4ceca5632 req-3370c857-e25d-4130-b7b0-275b1303d534 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0511591 May 24 10:09:52.471691 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-d8d939e2-a493-4b00-98b8-f8fa456e11af service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3237453 May 24 10:09:52.664480 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-deb18b2a-dfe0-4061-b425-1baa0ea3fb46 req-5da8cc98-49fe-45c8-b12f-76de71f34396 service neutron] 10.222.0.95 "GET /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1846640 May 24 10:09:52.668571 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:09:52.816465 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:52.836834 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1', 'network_id': '52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd', 'port_id': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1', 'mac_address': 'fa:16:3e:6d:db:6d', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 76, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '60925914-ac92-409e-94f9-b096645b63f1', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:09:52.899070 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:52.925519 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:09:52.945027 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:53.048665 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:53.050588 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:53.085864 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-998294e2-9a0a-497f-b09d-f4949ad07458 req-608fff96-eb07-4c0f-af4e-f0c4b21d43c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:53.089019 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-998294e2-9a0a-497f-b09d-f4949ad07458 req-608fff96-eb07-4c0f-af4e-f0c4b21d43c6 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 944 time: 0.0953488 May 24 10:09:53.122773 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:53.126275 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:53.138679 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-998294e2-9a0a-497f-b09d-f4949ad07458 req-e392f24a-ef7b-44ca-ba08-707e554dcd98 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0443902 May 24 10:09:53.328227 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['f24b7dd1-0def-4abf-89e7-2699ed5e0fe1'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:09:53.517525 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8af2050b-677f-4e83-a3be-213de19cf04f req-e819c0aa-41a9-41e6-9000-9ee35f00a209 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:53.519460 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8af2050b-677f-4e83-a3be-213de19cf04f req-e819c0aa-41a9-41e6-9000-9ee35f00a209 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1440504 May 24 10:09:53.596430 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8af2050b-677f-4e83-a3be-213de19cf04f req-56bfd36e-38f5-4fb0-a9ce-233b6f2fdafb tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0703862 May 24 10:09:54.267929 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b469299-2e51-44ba-86a6-fd1082c200a8 req-ffef1039-880a-4e76-93b9-429d4f2a59a4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:54.269760 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7b469299-2e51-44ba-86a6-fd1082c200a8 req-ffef1039-880a-4e76-93b9-429d4f2a59a4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1628160 May 24 10:09:54.276504 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-903f550b-a5f9-4713-a207-7a977773eb10 service neutron] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1295624 May 24 10:09:54.332280 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7b469299-2e51-44ba-86a6-fd1082c200a8 req-f2b6019b-6520-47a6-a444-c1d7b0595494 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0486789 May 24 10:09:54.417774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af348b7f-7176-480a-a996-8b30eb26c1c6 req-264c6fb2-33f1-4362-bc6d-029fb2eb5573 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:54.420391 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-af348b7f-7176-480a-a996-8b30eb26c1c6 req-264c6fb2-33f1-4362-bc6d-029fb2eb5573 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 945 time: 0.1488492 May 24 10:09:54.472840 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-af348b7f-7176-480a-a996-8b30eb26c1c6 req-0d48f0f9-f240-4907-b733-d52c13d24e6e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0463600 May 24 10:09:54.681979 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:09:54.888255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:09:54.893691 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:09:54.894062 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:55.027821 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:55.058718 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:55.070864 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:55.141775 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1', 'code': 200} May 24 10:09:55.169429 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:55.171236 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:09:55.213734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-37d30663-584e-4021-a9fb-ad61571e4372 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:55.395966 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:55.403338 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:55.495021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:55.554664 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:09:55.635392 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eea0c10d-cbe6-4e49-977c-53f2a0ea4ef0 req-bec7ce6f-9a4c-4070-bb45-5e4cfd9c4000 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:55.637083 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eea0c10d-cbe6-4e49-977c-53f2a0ea4ef0 req-bec7ce6f-9a4c-4070-bb45-5e4cfd9c4000 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1553161 May 24 10:09:55.642210 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45693c5e-9c70-4f56-b415-7bdb147415d8 req-a331cc54-1926-45d0-a5d5-395ed85a283c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:55.643723 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45693c5e-9c70-4f56-b415-7bdb147415d8 req-a331cc54-1926-45d0-a5d5-395ed85a283c service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 1099 time: 0.1041312 May 24 10:09:55.660161 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:09:55.663728 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:09:55.723321 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-eea0c10d-cbe6-4e49-977c-53f2a0ea4ef0 req-f48276fa-d4da-467e-b974-a9645b1a4e65 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0785804 May 24 10:09:55.725098 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45693c5e-9c70-4f56-b415-7bdb147415d8 req-8d21ebb1-9455-4e37-91e0-7fd5bdb4a5eb service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0641749 May 24 10:09:55.728353 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64538c17-91f7-4d12-aad5-13a32c46bc0a req-f460734c-6e70-4904-95af-72c76c59c837 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:55.731198 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-64538c17-91f7-4d12-aad5-13a32c46bc0a req-f460734c-6e70-4904-95af-72c76c59c837 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.1085870 May 24 10:09:55.762324 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=6']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:09:55.765675 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:09:55.802181 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-64538c17-91f7-4d12-aad5-13a32c46bc0a req-a26deffc-f275-462a-8a58-957c1b05c2e9 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0602005 May 24 10:09:57.076596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6184ed3b-b972-40ab-81ba-c541fb098eac req-379427c7-a131-45c3-a752-8e51c5307959 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.078142 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6184ed3b-b972-40ab-81ba-c541fb098eac req-379427c7-a131-45c3-a752-8e51c5307959 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.1075943 May 24 10:09:57.105968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78a7543b-1f67-4b89-a5d7-98e6e029fda8 req-58c140b3-586a-489d-8bf7-9937d3c29e1f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.107600 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-78a7543b-1f67-4b89-a5d7-98e6e029fda8 req-58c140b3-586a-489d-8bf7-9937d3c29e1f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0946851 May 24 10:09:57.132213 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6184ed3b-b972-40ab-81ba-c541fb098eac req-0cfdbd13-6135-4f1d-a2ea-85ca9dcddb70 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0454469 May 24 10:09:57.143339 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:09:57.164273 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-78a7543b-1f67-4b89-a5d7-98e6e029fda8 req-07c22884-87ed-45f1-a288-ef8d10db26f4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0504472 May 24 10:09:57.193786 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'code': 200} May 24 10:09:57.412044 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-80fe1d6a-64fb-401b-97b7-15d90e8b6574 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.413973 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-80fe1d6a-64fb-401b-97b7-15d90e8b6574 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d&tenant_id=ce96e2adbf10489aae7640c84f398b29 HTTP/1.1" status: 200 len: 1096 time: 0.0828736 May 24 10:09:57.525630 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-f25c7426-f4ea-4e6d-8279-e590da630284 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.527331 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-f25c7426-f4ea-4e6d-8279-e590da630284 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1096 time: 0.0794284 May 24 10:09:57.681920 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-b0dd49c4-a910-43a7-951f-d87f366665b2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.682226 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-b0dd49c4-a910-43a7-951f-d87f366665b2 service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.1467514 May 24 10:09:57.802737 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-1e132997-6c14-4c91-9c1c-89f3ee28501a service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 998 time: 0.1138749 May 24 10:09:57.873068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-04de0559-16d0-4dde-b775-88d1b7e27f83 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.874572 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-04de0559-16d0-4dde-b775-88d1b7e27f83 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.0639460 May 24 10:09:57.971051 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-a6a624c7-babd-4cec-b469-794def929c1e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:57.972582 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-a6a624c7-babd-4cec-b469-794def929c1e service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0903165 May 24 10:09:58.120043 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-f4db250d-6be5-4226-b724-d02063a072c7 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1394191 May 24 10:09:58.427064 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-459754aa-5700-459a-88c2-ba0266f3c299 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2978170 May 24 10:09:58.451911 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-da0d177d-c007-4ec8-a371-28393c4ca02a req-dfcc8f00-19e9-4dc2-bff1-d1b8426c420c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:58.453773 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-da0d177d-c007-4ec8-a371-28393c4ca02a req-dfcc8f00-19e9-4dc2-bff1-d1b8426c420c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.1799066 May 24 10:09:58.620408 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-da0d177d-c007-4ec8-a371-28393c4ca02a req-e2c1ff32-b2c0-4eba-9668-a68182695c3f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1600759 May 24 10:09:58.632640 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73b20086-9d30-4e17-acaa-634c5eda418f req-ccca5a12-3185-4ebb-965f-2d28fe0468de tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:58.634923 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-73b20086-9d30-4e17-acaa-634c5eda418f req-ccca5a12-3185-4ebb-965f-2d28fe0468de tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.2488163 May 24 10:09:58.686845 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-73b20086-9d30-4e17-acaa-634c5eda418f req-6f04b601-bb63-4439-ba80-92cbb5c90f43 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0449820 May 24 10:09:59.490409 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:09:59.901930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6171fd63-b556-4e10-bead-65943f6c00ff req-9c60d7c4-1beb-4fa0-bdcd-2829f374adfa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:09:59.903529 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6171fd63-b556-4e10-bead-65943f6c00ff req-9c60d7c4-1beb-4fa0-bdcd-2829f374adfa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0989118 May 24 10:09:59.960186 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-6171fd63-b556-4e10-bead-65943f6c00ff req-25f18dbb-72ce-486b-af73-69eb9503a2aa tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0457644 May 24 10:10:01.220567 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53e1463e-5d2a-4ead-9a9c-eb96e03e1fef req-d62f8eca-f104-45d1-a40e-ed4f79c692eb tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:01.222208 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-53e1463e-5d2a-4ead-9a9c-eb96e03e1fef req-d62f8eca-f104-45d1-a40e-ed4f79c692eb tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1379986 May 24 10:10:01.284722 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-53e1463e-5d2a-4ead-9a9c-eb96e03e1fef req-961af7ab-2144-437c-8678-5e71144b87f4 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0558500 May 24 10:10:02.574444 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4decd05-d442-4ae0-b864-fbf008567d18 req-7bb8d4f1-857d-45be-b944-6875164396bc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:02.576144 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e4decd05-d442-4ae0-b864-fbf008567d18 req-7bb8d4f1-857d-45be-b944-6875164396bc tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1398129 May 24 10:10:02.643518 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e4decd05-d442-4ae0-b864-fbf008567d18 req-a7676919-a920-4e1a-bd61-20b0677fa62a tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0567117 May 24 10:10:03.340735 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3018c2de-34c9-4f93-9526-dd7283972164 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:03.854080 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5d88956-0eca-409d-bdb1-f5c5eef28f4e req-4e827d11-f24c-45ac-9eda-d7ce43d8a836 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:03.855651 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c5d88956-0eca-409d-bdb1-f5c5eef28f4e req-4e827d11-f24c-45ac-9eda-d7ce43d8a836 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.0878372 May 24 10:10:03.913681 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c5d88956-0eca-409d-bdb1-f5c5eef28f4e req-ad3d03c0-7571-4ef1-b878-4b026d74411e tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0467322 May 24 10:10:04.735864 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:10:04.893120 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:04.908242 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'port_id': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'mac_address': 'fa:16:3e:03:bf:e8', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 81, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:10:04.966989 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:04.985810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b5308ea7-0517-43e4-830b-7fd131b3a9c7 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:10:05.004375 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:05.095039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:05.099961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:05.154520 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-902ce27f-6068-4a71-a4e4-bfba0e8634a0 req-e561a73a-7acb-4295-97cf-9b27abee5e28 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:05.156991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-902ce27f-6068-4a71-a4e4-bfba0e8634a0 req-e561a73a-7acb-4295-97cf-9b27abee5e28 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 943 time: 0.1130638 May 24 10:10:05.187424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=7']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:05.196348 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.197709 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.197709 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.198021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.198341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.198739 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.199083 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.199599 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.200133 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.200658 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.201224 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.201769 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:05.206268 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:05.211237 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-902ce27f-6068-4a71-a4e4-bfba0e8634a0 req-a5744828-ce2b-4a90-bf6e-25dfe4d2886b tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0466514 May 24 10:10:05.364867 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['b5308ea7-0517-43e4-830b-7fd131b3a9c7'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:10:05.896020 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7b5164c1-b22d-4d8c-8833-3bd0b03881dd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:05.928046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:06.424330 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbf87f82-a8bf-49f2-b26b-8330506a7f46 req-bce84fe7-be50-43c8-8246-bf83b7eb4cff tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:06.426219 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cbf87f82-a8bf-49f2-b26b-8330506a7f46 req-bce84fe7-be50-43c8-8246-bf83b7eb4cff tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 944 time: 0.0951812 May 24 10:10:06.479804 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cbf87f82-a8bf-49f2-b26b-8330506a7f46 req-9a3c7e3e-3d6b-470d-982c-9c041457978f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0477366 May 24 10:10:06.748617 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:10:06.838671 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port b5308ea7-0517-43e4-830b-7fd131b3a9c7 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:10:06.839057 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:07.036567 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:07.083646 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:10:07.096302 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:07.140943 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'code': 200} May 24 10:10:07.173388 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:07.177477 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-809e1c32-1e9f-4ae7-a696-9a1859b5b35b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:07.248704 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:07.318448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=8']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:07.325867 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:07.578482 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e27b99a0-1389-47ad-a86f-a025b833e95c req-9326f579-d74d-4521-8448-68e6b2d7ec6c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:07.581691 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e27b99a0-1389-47ad-a86f-a025b833e95c req-9326f579-d74d-4521-8448-68e6b2d7ec6c service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.0937800 May 24 10:10:07.635043 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e27b99a0-1389-47ad-a86f-a025b833e95c req-93d6176d-8313-468f-950d-22c232ccf239 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0460963 May 24 10:10:07.722696 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79146e92-8a86-400d-8843-e3de442b7b48 req-07cc2a13-1773-4e83-941d-a394ff825f86 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:07.724695 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79146e92-8a86-400d-8843-e3de442b7b48 req-07cc2a13-1773-4e83-941d-a394ff825f86 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1012087 May 24 10:10:07.783042 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-79146e92-8a86-400d-8843-e3de442b7b48 req-bf142e71-374d-404e-871f-ffd719e4e8c9 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0512795 May 24 10:10:08.780075 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:09.009421 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57f32a80-1aa6-4518-89fc-f88a4ea9e815 req-8acb482c-f670-4830-a81b-a474af01d3a5 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:09.011955 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-57f32a80-1aa6-4518-89fc-f88a4ea9e815 req-8acb482c-f670-4830-a81b-a474af01d3a5 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1013517 May 24 10:10:09.078570 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-57f32a80-1aa6-4518-89fc-f88a4ea9e815 req-fb0702ae-7a05-451e-a0b4-b1f67f221700 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0582106 May 24 10:10:15.693627 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a16733d3-91e4-4f0a-a944-60a7634af793 req-5e9a0e5e-a31f-4149-9b4e-f5d600845465 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:15.695358 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a16733d3-91e4-4f0a-a944-60a7634af793 req-5e9a0e5e-a31f-4149-9b4e-f5d600845465 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.0913458 May 24 10:10:15.752310 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a16733d3-91e4-4f0a-a944-60a7634af793 req-280e44ba-d826-4893-8055-0f01c800cdea tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0461676 May 24 10:10:16.968513 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a31db4f7-e388-4702-a4fb-0038edae6734 req-cb95ea6a-5f95-4930-a20a-6d31645450a0 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:16.971101 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a31db4f7-e388-4702-a4fb-0038edae6734 req-cb95ea6a-5f95-4930-a20a-6d31645450a0 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.0965507 May 24 10:10:17.025353 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a31db4f7-e388-4702-a4fb-0038edae6734 req-ae16497c-8fcb-4791-ba5b-695d9c479081 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0483320 May 24 10:10:17.622821 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-98435781-fa8a-4769-923c-023019ea68be tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:17.624445 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-98435781-fa8a-4769-923c-023019ea68be tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.1010022 May 24 10:10:17.807340 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Deleting port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:10:17.807604 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:17.968193 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] For port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:17.968706 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:18.109239 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Calling delete_port for f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:10:18.335067 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e9fe623-4281-4545-9403-d874bdd66bc7 req-10404c5a-539e-4385-9d25-c589cc307c7f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:18.336628 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6e9fe623-4281-4545-9403-d874bdd66bc7 req-10404c5a-539e-4385-9d25-c589cc307c7f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 946 time: 0.1168647 May 24 10:10:18.373726 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:18.386229 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:18.390780 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d', '8868b74e-802d-457a-bde2-4bd4e630837c', 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:10:18.418031 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6e9fe623-4281-4545-9403-d874bdd66bc7 req-5a3a09e1-4ccf-4921-b235-005729c5736c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0639763 May 24 10:10:18.470597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event deleted for resources: {'Port': ['ID=f24b7dd1-0def-4abf-89e7-2699ed5e0fe1,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:18.519594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.519993 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.520505 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.520889 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.521291 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.521660 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.522028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.522404 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.522758 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.523116 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.523654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.524018 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:18.532858 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:18.574807 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] No DVR routers for this DVR port f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:10:18.578273 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:18.588306 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'name': 'network-vif-deleted', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:10:18.662256 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0e8beb81-ad4f-4a82-be8e-55512f7ed570', 'name': 'network-vif-deleted', 'tag': 'f24b7dd1-0def-4abf-89e7-2699ed5e0fe1', 'status': 'completed', 'code': 200} May 24 10:10:18.665498 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a666f3b2-8783-49cd-bb35-052c4c6aca1e req-f5798938-3b00-43ab-8a1f-0fa5d9a4534d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "DELETE /v2.0/ports/f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 HTTP/1.1" status: 204 len: 173 time: 1.0326538 May 24 10:10:19.655388 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ba140273-8ec3-4148-8bfd-63b22dd7cc51 req-0c5df43b-0c41-40ea-acae-fe171c886834 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=0e8beb81-ad4f-4a82-be8e-55512f7ed570 HTTP/1.1" status: 200 len: 210 time: 0.0738392 May 24 10:10:20.197034 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:20.812211 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:10:20.858176 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:10:20.858538 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f24b7dd1-0def-4abf-89e7-2699ed5e0fe1 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:10:22.914349 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9edbb05a-47d5-4d81-b87f-12fc2c584fb3 req-c11fa0d4-03aa-4820-9fbe-d107ebf604a4 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:22.915503 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-9edbb05a-47d5-4d81-b87f-12fc2c584fb3 req-c11fa0d4-03aa-4820-9fbe-d107ebf604a4 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:10:23.013817 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-9edbb05a-47d5-4d81-b87f-12fc2c584fb3 req-c11fa0d4-03aa-4820-9fbe-d107ebf604a4 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:10:23.015101 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9edbb05a-47d5-4d81-b87f-12fc2c584fb3 req-c11fa0d4-03aa-4820-9fbe-d107ebf604a4 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 945 time: 0.2115073 May 24 10:10:23.075428 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9edbb05a-47d5-4d81-b87f-12fc2c584fb3 req-aeaa532b-37e1-4cbc-bcb5-397e13a7c7eb tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0539060 May 24 10:10:24.297378 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29719d31-00e6-41a5-96d2-957596e95db2 req-9b831b2f-74f1-49f8-afc2-a0fec97f6fca tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:24.299226 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-29719d31-00e6-41a5-96d2-957596e95db2 req-9b831b2f-74f1-49f8-afc2-a0fec97f6fca tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 945 time: 0.0931513 May 24 10:10:24.349998 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-29719d31-00e6-41a5-96d2-957596e95db2 req-3db55f3b-a684-4c29-b53d-f3b5a9a7c7e8 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0440438 May 24 10:10:24.443363 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-74711f5a-18bf-491f-8937-c99b601924f1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:24.445037 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-74711f5a-18bf-491f-8937-c99b601924f1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 945 time: 0.0892787 May 24 10:10:24.704591 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Deleting port aeb4b23e-f182-4da2-a23f-751d3b29c315 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:10:24.704889 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:24.860587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] For port aeb4b23e-f182-4da2-a23f-751d3b29c315, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=aeb4b23e-f182-4da2-a23f-751d3b29c315,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:24.860932 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:24.992226 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Calling delete_port for aeb4b23e-f182-4da2-a23f-751d3b29c315 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:10:25.179471 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:25.185675 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:25.192322 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d', '8868b74e-802d-457a-bde2-4bd4e630837c', 'aeb4b23e-f182-4da2-a23f-751d3b29c315'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:10:25.234915 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event deleted for resources: {'Port': ['ID=aeb4b23e-f182-4da2-a23f-751d3b29c315,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:25.238195 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:25.321534 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] No DVR routers for this DVR port aeb4b23e-f182-4da2-a23f-751d3b29c315 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:10:25.325019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:25.329522 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'name': 'network-vif-deleted', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:10:25.410016 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5d2ef85f-1bb0-41c4-bfab-d49ac6f59185', 'name': 'network-vif-deleted', 'tag': 'aeb4b23e-f182-4da2-a23f-751d3b29c315', 'status': 'completed', 'code': 200} May 24 10:10:25.417433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5e7f10a5-4503-4a92-9f56-fe694e6c9259 req-9e517537-312f-4465-9f9e-656d3694359c tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "DELETE /v2.0/ports/aeb4b23e-f182-4da2-a23f-751d3b29c315 HTTP/1.1" status: 204 len: 173 time: 0.9662368 May 24 10:10:25.564767 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f35db40-deca-4bd7-a095-68ebb6c137d5 req-ad31a8c2-4061-408e-9336-e9fa490e7604 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=5d2ef85f-1bb0-41c4-bfab-d49ac6f59185 HTTP/1.1" status: 200 len: 210 time: 0.0664141 May 24 10:10:27.533547 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f208158-3484-4c11-adac-4014caf43720 req-5aef6f66-f0af-45de-87f8-3684c37a9ee1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:27.534791 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-1f208158-3484-4c11-adac-4014caf43720 req-5aef6f66-f0af-45de-87f8-3684c37a9ee1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:10:27.636578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-1f208158-3484-4c11-adac-4014caf43720 req-5aef6f66-f0af-45de-87f8-3684c37a9ee1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:10:27.638021 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1f208158-3484-4c11-adac-4014caf43720 req-5aef6f66-f0af-45de-87f8-3684c37a9ee1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 945 time: 0.2089219 May 24 10:10:27.691486 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1f208158-3484-4c11-adac-4014caf43720 req-de4cfd61-3df9-4c93-a667-faeab451b930 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0463123 May 24 10:10:28.839018 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device aeb4b23e-f182-4da2-a23f-751d3b29c315 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:10:28.897361 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: aeb4b23e-f182-4da2-a23f-751d3b29c315 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:10:28.897950 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device aeb4b23e-f182-4da2-a23f-751d3b29c315 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:10:29.103671 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-a3948c8a-781e-4c71-95a3-04b7d42ae537 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:29.106158 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-a3948c8a-781e-4c71-95a3-04b7d42ae537 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 945 time: 0.1589079 May 24 10:10:29.169855 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6319986-c0c7-4849-99be-e0ec2fca4a68 req-634dfd56-d0ec-4ac7-862a-d1fe996435ed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:29.171456 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b6319986-c0c7-4849-99be-e0ec2fca4a68 req-634dfd56-d0ec-4ac7-862a-d1fe996435ed tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 945 time: 0.3051820 May 24 10:10:29.233911 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b6319986-c0c7-4849-99be-e0ec2fca4a68 req-bddf3c70-e7ae-4079-bc37-e016571ab1cb tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0509856 May 24 10:10:29.285780 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Deleting port 8868b74e-802d-457a-bde2-4bd4e630837c {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:10:29.286131 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:29.436261 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] For port 8868b74e-802d-457a-bde2-4bd4e630837c, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=8868b74e-802d-457a-bde2-4bd4e630837c,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:29.436563 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:29.570007 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Calling delete_port for 8868b74e-802d-457a-bde2-4bd4e630837c owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:10:29.801930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:29.808765 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d2e3b552-aa27-4e3d-8f4f-08a9011d3c8a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:29.815545 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d', '8868b74e-802d-457a-bde2-4bd4e630837c'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:10:29.860884 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event deleted for resources: {'Port': ['ID=8868b74e-802d-457a-bde2-4bd4e630837c,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:29.864652 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:29.942360 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] No DVR routers for this DVR port 8868b74e-802d-457a-bde2-4bd4e630837c on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:10:29.944894 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:29.948667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'name': 'network-vif-deleted', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:10:30.036205 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '39724a92-c826-4944-8a07-916d46da0e0d', 'name': 'network-vif-deleted', 'tag': '8868b74e-802d-457a-bde2-4bd4e630837c', 'status': 'completed', 'code': 200} May 24 10:10:30.048555 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47a9a790-794a-454f-8a0e-b708ecb5482b req-bc823ea8-6867-41e4-961d-e504f9e52ef1 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "DELETE /v2.0/ports/8868b74e-802d-457a-bde2-4bd4e630837c HTTP/1.1" status: 204 len: 173 time: 0.9362857 May 24 10:10:30.423004 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-34b1d768-eed5-4a6f-981f-2290197b4a87 req-a5646515-a1ea-41bf-9905-c4632b275b5d tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=39724a92-c826-4944-8a07-916d46da0e0d HTTP/1.1" status: 200 len: 210 time: 0.0475745 May 24 10:10:32.879312 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 8868b74e-802d-457a-bde2-4bd4e630837c no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:10:32.915892 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 8868b74e-802d-457a-bde2-4bd4e630837c {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:10:32.916294 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 8868b74e-802d-457a-bde2-4bd4e630837c not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:10:33.323022 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c0dd211a-c9e5-41f1-9c18-9f3dabf5c82a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:33.638219 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b2ac2235-b36b-48d8-8d7e-f1fe80222569 req-a1a327ef-f391-44b1-961a-3cfa37567f04 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:33.638856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-b2ac2235-b36b-48d8-8d7e-f1fe80222569 req-a1a327ef-f391-44b1-961a-3cfa37567f04 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:10:33.750362 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-b2ac2235-b36b-48d8-8d7e-f1fe80222569 req-a1a327ef-f391-44b1-961a-3cfa37567f04 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:10:33.751682 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2ac2235-b36b-48d8-8d7e-f1fe80222569 req-a1a327ef-f391-44b1-961a-3cfa37567f04 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.2703917 May 24 10:10:33.801607 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2ac2235-b36b-48d8-8d7e-f1fe80222569 req-271e1274-9d37-48db-b9c8-eb62a6bf48b8 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441458 May 24 10:10:35.021978 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7da820e6-88c9-4e2a-9508-12eaa5cb0c0f req-b28fd891-4d66-4856-ada4-3d0c42661e0a tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:35.023679 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7da820e6-88c9-4e2a-9508-12eaa5cb0c0f req-b28fd891-4d66-4856-ada4-3d0c42661e0a tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.1001806 May 24 10:10:35.077957 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7da820e6-88c9-4e2a-9508-12eaa5cb0c0f req-ff843c76-919a-4ac1-a681-88d7ba715fa8 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434656 May 24 10:10:35.519902 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-4c003638-9999-4a76-bb89-676f541e820e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:35.521570 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-4c003638-9999-4a76-bb89-676f541e820e tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.0910466 May 24 10:10:35.790607 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Deleting port 3768267d-8ff1-4093-bb55-85e4456cd30d {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:10:35.791586 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:35.809503 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-55b76bb8-61e5-407d-8352-37c2ca6380af None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:35.871525 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:35.943717 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] For port 3768267d-8ff1-4093-bb55-85e4456cd30d, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=3768267d-8ff1-4093-bb55-85e4456cd30d,segment=NetworkSegment(b42a154a-6873-4e20-9753-b2afc6ed11b0),segment_id=b42a154a-6873-4e20-9753-b2afc6ed11b0)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:10:35.944084 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:36.093433 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Calling delete_port for 3768267d-8ff1-4093-bb55-85e4456cd30d owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:10:36.365896 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e72e991d-a1e5-4d14-a8fd-7e733e0f776f req-86eb3f55-12fe-4792-af8b-9cee7418c978 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:36.367560 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e72e991d-a1e5-4d14-a8fd-7e733e0f776f req-86eb3f55-12fe-4792-af8b-9cee7418c978 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 945 time: 0.1736114 May 24 10:10:36.374366 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:36.385632 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:36.386975 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['3768267d-8ff1-4093-bb55-85e4456cd30d'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:10:36.452073 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Pushing event deleted for resources: {'Port': ['ID=3768267d-8ff1-4093-bb55-85e4456cd30d,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:36.454855 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:36.489932 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:10:36.566478 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e72e991d-a1e5-4d14-a8fd-7e733e0f776f req-df09f9af-ff35-4dd7-9d00-803160abd611 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/security-groups?id=b7676221-dbea-47ff-b118-72910e441b53&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1840954 May 24 10:10:36.567636 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] No DVR routers for this DVR port 3768267d-8ff1-4093-bb55-85e4456cd30d on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:10:36.570903 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:36.574150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'name': 'network-vif-deleted', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:10:36.635320 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '09bb9b57-7d05-4ff0-8c0e-5b7eac74665a', 'name': 'network-vif-deleted', 'tag': '3768267d-8ff1-4093-bb55-85e4456cd30d', 'status': 'completed', 'code': 200} May 24 10:10:36.660793 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c6d0a734-d51d-43e3-b0f7-ce5717d99211 req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.95 "DELETE /v2.0/ports/3768267d-8ff1-4093-bb55-85e4456cd30d HTTP/1.1" status: 204 len: 173 time: 1.1286597 May 24 10:10:37.224471 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:37.767113 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d667e88c-f38f-4fea-97f3-6dd65dc67662 req-d0cc9f88-2723-43fd-b742-42af5fa8a063 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Synchronizing usage tracker for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:10:37.870899 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d667e88c-f38f-4fea-97f3-6dd65dc67662 req-d0cc9f88-2723-43fd-b742-42af5fa8a063 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Unset dirty status for tenant:8e85a9260aad4c93aa21262f8764f011 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:10:37.872549 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d667e88c-f38f-4fea-97f3-6dd65dc67662 req-d0cc9f88-2723-43fd-b742-42af5fa8a063 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] 10.222.0.67 "GET /v2.0/ports?device_id=09bb9b57-7d05-4ff0-8c0e-5b7eac74665a HTTP/1.1" status: 200 len: 210 time: 0.1502235 May 24 10:10:38.796535 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:38.988032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33268) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:39.097915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Deleting subnet 60925914-ac92-409e-94f9-b096645b63f1 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:10:39.103251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:39.198733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:39.306748 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:39.705405 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:39.710027 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-99cf11a0-1a4a-4f87-92b3-fc14f385a98c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:39.747795 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=60925914-ac92-409e-94f9-b096645b63f1,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:39.754997 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:39.763898 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.764282 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.764693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.765073 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.765440 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.765793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.766140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.766496 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.766845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.767221 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.767532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.767876 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:10:39.774383 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:39.780277 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/60925914-ac92-409e-94f9-b096645b63f1 HTTP/1.1" status: 204 len: 168 time: 0.7902865 May 24 10:10:39.782537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33272) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:39.898282 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.133820 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.148531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:10:40.163229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'b42a154a-6873-4e20-9753-b2afc6ed11b0') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:10:40.163688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'b42a154a-6873-4e20-9753-b2afc6ed11b0') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:10:40.177475 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.196567 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.203758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Releasing vlan 76 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:10:40.204212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:10:40.218901 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:10:40.390554 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.394885 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:40.410358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:40.422612 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Pushing event deleted for resources: {'Network': ['ID=52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:40.426393 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:40.488592 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-7567b9ca-edef-42b0-b35a-6a76f71e2855 admin admin] 10.222.0.67 "DELETE /v2.0/networks/52ca9fee-c69b-4d5c-b7bb-52812f9c5bbd HTTP/1.1" status: 204 len: 168 time: 0.7048130 May 24 10:10:40.923319 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 3768267d-8ff1-4093-bb55-85e4456cd30d no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:10:40.959790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 3768267d-8ff1-4093-bb55-85e4456cd30d {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:10:40.960148 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 3768267d-8ff1-4093-bb55-85e4456cd30d not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:10:40.988193 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33280) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:41.048927 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c3a9ab35-caf9-4e1f-bccb-5a19f3a14b66 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=8e85a9260aad4c93aa21262f8764f011&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0593140 May 24 10:10:41.051954 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33284) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:41.119744 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:41.150730 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:41.282327 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:41.286938 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0a2695ff-a5fa-464a-8151-c3be8ae3fca5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:41.301951 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b7676221-dbea-47ff-b118-72910e441b53,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:41.304472 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:41.390489 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-2da20989-6527-4259-b419-954b00293776 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:41.391667 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-2da20989-6527-4259-b419-954b00293776 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/b7676221-dbea-47ff-b118-72910e441b53 HTTP/1.1" status: 204 len: 168 time: 0.3387051 May 24 10:10:47.566569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33348) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:47.699859 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:47.717071 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:10:47.721532 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Usage tracker for resource:security_group_rule and tenant:d41c0ff990c44df58f7719fc391264fb is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:10:47.724784 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:10:47.731371 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:10:47.833920 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:47.881572 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:47.890064 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a55db03c-24c0-4f60-8c33-cff01f6d6ef8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:47.935664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cc512969-6816-489a-b6a2-3c28bec5cf56,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:47.940060 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=d41c0ff990c44df58f7719fc391264fb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3723748 May 24 10:10:47.941083 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1f8319de-be93-4b7e-b1c4-acf93243415f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:47.943571 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33354) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:48.016647 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:48.049254 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:48.213834 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:48.217923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d4f6e424-3ad9-4098-8d19-b55344e7e9f7 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:10:48.232290 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cc512969-6816-489a-b6a2-3c28bec5cf56,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:10:48.235105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:10:48.239878 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:48.241831 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/cc512969-6816-489a-b6a2-3c28bec5cf56 HTTP/1.1" status: 204 len: 168 time: 0.2970755 May 24 10:10:50.224448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:51.618178 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-1ca479f1-59b4-4102-936c-a7d762c834ad service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2388780 May 24 10:10:51.806147 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-9c50c00c-f641-484d-b133-833f90e92f3b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:51.808838 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-9c50c00c-f641-484d-b133-833f90e92f3b service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.1144016 May 24 10:10:52.013935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-4dc43a16-609b-4123-b3c0-df5c86b7f423 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:52.015433 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-4dc43a16-609b-4123-b3c0-df5c86b7f423 service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.1728017 May 24 10:10:52.137113 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-2a1c2a61-bb17-4a58-91ba-34bf2cbfcd82 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 1000 time: 0.1140182 May 24 10:10:52.211578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-0d6ca334-06c3-453f-a565-32267c300057 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:52.213092 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-0d6ca334-06c3-453f-a565-32267c300057 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.0692439 May 24 10:10:52.307323 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-d6ccc2c6-ff86-4d33-9d97-aa100cb21a52 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:10:52.308775 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-d6ccc2c6-ff86-4d33-9d97-aa100cb21a52 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0890412 May 24 10:10:52.321099 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-088d631d-942d-4465-90d9-887747e51fe6 service neutron] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:10:52.322182 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-088d631d-942d-4465-90d9-887747e51fe6 service neutron] 10.222.0.95 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0067179 May 24 10:10:52.584300 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-01cc235c-28f8-419c-b1cc-1b51656f4ed0 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2543564 May 24 10:10:52.739036 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-05c38e72-d95c-40dd-9959-6acb00ca5a87 req-e408b5b7-2bb3-46cd-8a97-8d65a873545d service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1478944 May 24 10:10:59.662597 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33482) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:10:59.820920 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:10:59.837659 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:10:59.842099 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Usage tracker for resource:security_group_rule and tenant:cef5868dd199431394cf6e76a68f0464 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:10:59.849200 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:10:59.853533 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:10:59.952749 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.007049 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.015831 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8a95f889-dca3-40c5-941b-ec1a4791e322 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:00.066446 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=cef5868dd199431394cf6e76a68f0464&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4021797 May 24 10:11:00.067264 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=68971a39-2f6c-42c7-8ad8-bee991d2cde9,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:00.070587 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33488) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:00.078471 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1d48e417-2fb4-41db-b291-eb77387b4042 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:00.158725 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.193949 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.332952 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.337895 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-47323767-5a34-484e-86d2-682e0f41b158 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:00.352555 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=68971a39-2f6c-42c7-8ad8-bee991d2cde9,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:00.355148 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:00.359775 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:00.361394 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/68971a39-2f6c-42c7-8ad8-bee991d2cde9 HTTP/1.1" status: 204 len: 168 time: 0.2890980 May 24 10:11:01.090047 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33496) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:01.109720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.126479 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:11:01.131078 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Usage tracker for resource:security_group_rule and tenant:6dba2bba2f9743aca364c68c6cf3747a is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:11:01.134566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:11:01.141521 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:11:01.243635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.367813 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.377888 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d91c054b-228e-483d-a333-76009987d7a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:01.425801 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6dba2bba2f9743aca364c68c6cf3747a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3342898 May 24 10:11:01.426648 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8f13345d-9655-4deb-a8f6-8d32bdc97f99,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:01.429004 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33500) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:01.431341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:01.498745 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.529721 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.687185 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.691663 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-964c389f-124a-4ebd-8cd1-9063408c7361 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:01.712935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8f13345d-9655-4deb-a8f6-8d32bdc97f99,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:01.718582 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:01.727195 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:01.730371 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-8cdd5973-4483-4653-8452-18057d415a71 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/8f13345d-9655-4deb-a8f6-8d32bdc97f99 HTTP/1.1" status: 204 len: 168 time: 0.2988670 May 24 10:11:03.395628 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b46d76e4-3b65-4c38-b87e-06cde37f0113 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.223361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33544) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:04.436578 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.455112 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:11:04.459492 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Usage tracker for resource:security_group_rule and tenant:32adda182f974520af01c88f38e207f2 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:11:04.462847 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:11:04.469208 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:11:04.599596 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.664755 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.678624 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0467dc75-a126-4ea4-a238-bf4007785aae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:04.726764 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=32adda182f974520af01c88f38e207f2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5014219 May 24 10:11:04.729479 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c1e0ac91-b35b-4b4b-ab1a-baa30a46c74d,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:04.730635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33550) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:04.734965 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-a8f573b0-37af-42ac-a5ed-d16e51e8d0f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:04.807297 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.838212 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.995544 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:04.999974 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c335b472-9aab-4f5c-968b-04f7c0d4c891 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:05.015129 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c1e0ac91-b35b-4b4b-ab1a-baa30a46c74d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:05.018061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:05.028579 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:05.030046 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-e8fc630f-4b27-4a81-a008-23dc904c5ff6 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/c1e0ac91-b35b-4b4b-ab1a-baa30a46c74d HTTP/1.1" status: 204 len: 168 time: 0.2980437 May 24 10:11:05.841527 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-be89c9c2-c033-4fd2-ac70-505e8fdd3a53 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:05.892740 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:07.247125 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:08.875291 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:11.798492 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33610) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:11.934222 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:11.950389 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:11:11.954959 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Usage tracker for resource:security_group_rule and tenant:6897c082efe34f729b33c759d4106602 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:11:11.958437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:11:11.965091 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:11:12.063381 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.138102 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.146337 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4f308a65-eebb-440c-8af8-d2bca72ce6aa admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:12.199993 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6897c082efe34f729b33c759d4106602&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4003465 May 24 10:11:12.200924 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=be0d1283-ef35-4806-b31e-4f1425abc951,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:12.204259 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33616) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:12.205163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:12.279294 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.310486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.452758 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.458330 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4464bb77-652d-4354-b44a-08c8ce13df8a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:12.476192 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=be0d1283-ef35-4806-b31e-4f1425abc951,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:12.481096 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:12.483440 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:12.484983 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-c2498f84-1552-4279-85c3-4ab2d3074b54 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/be0d1283-ef35-4806-b31e-4f1425abc951 HTTP/1.1" status: 204 len: 168 time: 0.2786999 May 24 10:11:13.491125 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:11:13.620646 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33624) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:13.640439 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:13.656969 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:11:13.661480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Usage tracker for resource:security_group_rule and tenant:6efe55c6dff14ad59aa7fba6e135f40a is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:11:13.665023 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:11:13.671774 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:11:13.774133 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:13.823487 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:13.833207 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-389fbb6e-7009-40b5-9513-e7e28d1d6857 tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:13.876471 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=6efe55c6dff14ad59aa7fba6e135f40a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2543573 May 24 10:11:13.878638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=81498039-9bb7-4c3e-a549-26dbd50ac613,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:13.882981 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33628) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:13.883820 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:13.994690 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:14.044743 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:14.189647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:14.196213 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0bcd5fd7-6b73-45a3-a9af-f849ccb374f3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:14.213688 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=81498039-9bb7-4c3e-a549-26dbd50ac613,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:14.216442 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:14.224892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:14.226944 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-87fb4e03-7e7d-4e48-a34d-9d05132fe173 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/81498039-9bb7-4c3e-a549-26dbd50ac613 HTTP/1.1" status: 204 len: 168 time: 0.3455513 May 24 10:11:20.217800 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:29.873830 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33692) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:30.432813 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Deleting port 12af85ea-363f-46d1-a661-0c5c82780e12 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:30.433211 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:30.584711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:30.585165 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:30.798277 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Calling delete_port for 12af85ea-363f-46d1-a661-0c5c82780e12 owned by network:floatingip {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:31.068719 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:31.077896 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-eec0c8cf-260c-4566-97ce-8b36ca67a787 tempest-VolumesNegativeTest-766272021 tempest-VolumesNegativeTest-766272021-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:31.153264 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event deleted for resources: {'Port': ['ID=12af85ea-363f-46d1-a661-0c5c82780e12,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:31.184993 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:31.260006 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for floatingip, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:31.261682 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Floating IP 4db86f31-88c4-47f1-a287-fac820ccc2bf disassociated (deleted). External IP: 172.27.0.95, port: b5308ea7-0517-43e4-830b-7fd131b3a9c7. May 24 10:11:31.768711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Router 91d55ed4-1190-40a0-83e1-ece622fa9c2e has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:11:31.975231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:11:31.988805 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Sync routers for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:11:32.000886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:32.001566 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:32.052633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:32.153415 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "DELETE /v2.0/floatingips/4db86f31-88c4-47f1-a287-fac820ccc2bf HTTP/1.1" status: 204 len: 168 time: 2.2780118 May 24 10:11:32.156016 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:11:32.286899 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:32.508686 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'status': 'completed', 'code': 200} May 24 10:11:32.658093 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-d72de0ea-146e-4783-96c1-726fd7e181d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:32.659919 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-d72de0ea-146e-4783-96c1-726fd7e181d8 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=ce96e2adbf10489aae7640c84f398b29&device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 1098 time: 0.1204493 May 24 10:11:32.857995 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-f0f4296c-a5dd-4b70-bce2-25c0886a9833 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:32.860616 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-f0f4296c-a5dd-4b70-bce2-25c0886a9833 service neutron] 10.222.0.95 "GET /v2.0/networks?id=9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 200 len: 909 time: 0.1930139 May 24 10:11:33.013677 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-6ec9cf7c-014e-4679-bda9-6c0c0f993bc7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 200 len: 217 time: 0.1429019 May 24 10:11:33.017146 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:33.024571 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Checking router: 91d55ed4-1190-40a0-83e1-ece622fa9c2e for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:11:33.039492 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:33.040148 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:33.219941 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-832e83d9-4453-4875-b016-8af9d8417566 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:33.222303 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-832e83d9-4453-4875-b016-8af9d8417566 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 200 len: 848 time: 0.1535161 May 24 10:11:33.305000 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8fcade31-5aa0-4a1a-b05c-3d5193cb4834 req-463740c8-05bd-4c58-af41-24555391ce44 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:33.306604 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8fcade31-5aa0-4a1a-b05c-3d5193cb4834 req-463740c8-05bd-4c58-af41-24555391ce44 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.3206038 May 24 10:11:33.375986 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-8fcade31-5aa0-4a1a-b05c-3d5193cb4834 req-6616340a-149d-477d-b360-f167287913ea tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0584242 May 24 10:11:33.379168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-fd0698c1-c56e-4484-ada5-2ef5331ee584 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:33.380661 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-fd0698c1-c56e-4484-ada5-2ef5331ee584 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=9a3ab75c-0613-43fc-b406-9209bc3e8724&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1477964 May 24 10:11:33.382126 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:33.443069 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b03e5ee6-1902-4ed2-b2a5-d54eccf098ad None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:33.748543 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:33.753785 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4c09f703-27fb-4a66-8a96-ecc6b58ecd7d None None] The sync data for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi is: [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:06:05Z', 'revision_number': 4, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'c366b56b-4d33-4772-8a44-91c29911caf1', 'name': '', 'network_id': '9a3ab75c-0613-43fc-b406-9209bc3e8724', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'mac_address': 'fa:16:3e:3a:43:c0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 478, 'fixed_ips': [{'subnet_id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:04Z', 'updated_at': '2021-05-24T10:06:13Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'subnets': [{'id': '0f050b21-5ec6-4623-a5c0-4e203399a27b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:11:33.984962 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a96ce0ec-1f97-4ff8-bccb-6fa525279ba3 None None] New status for floating IP 4db86f31-88c4-47f1-a287-fac820ccc2bf: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:11:34.425273 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-edb0d0ce-f8ab-4daa-b8a8-12f3b9e46af0 service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1418262 May 24 10:11:34.643143 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17b60890-64ab-4ab1-aaba-166713ad9a33 req-7f7b972a-d1c2-4d86-9dea-94cd9d0b74df tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:34.644648 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-17b60890-64ab-4ab1-aaba-166713ad9a33 req-7f7b972a-d1c2-4d86-9dea-94cd9d0b74df tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:11:34.737231 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-17b60890-64ab-4ab1-aaba-166713ad9a33 req-7f7b972a-d1c2-4d86-9dea-94cd9d0b74df tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:11:34.738645 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-17b60890-64ab-4ab1-aaba-166713ad9a33 req-7f7b972a-d1c2-4d86-9dea-94cd9d0b74df tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.2243836 May 24 10:11:34.795997 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-17b60890-64ab-4ab1-aaba-166713ad9a33 req-b60317d1-1198-41c5-9cee-9b413805c35b tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0502169 May 24 10:11:35.299378 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-2ae83048-fde9-4f82-8b99-7a2f19911743 req-10fef7f1-d8cb-49a9-a958-e0019d47034d service neutron] 10.222.0.95 "GET /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1415319 May 24 10:11:35.789950 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-469b53e6-0652-493e-8a91-bf33477f1ad1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:35.874708 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:36.011840 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba1d4b19-5b2e-47de-8cb7-791e63e7823e req-8339d5d4-13dd-4cf7-b42d-4eac6e1f6242 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:36.013515 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ba1d4b19-5b2e-47de-8cb7-791e63e7823e req-8339d5d4-13dd-4cf7-b42d-4eac6e1f6242 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.0899186 May 24 10:11:36.066604 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ba1d4b19-5b2e-47de-8cb7-791e63e7823e req-17e16bc7-c2c7-4349-b287-5103b4431810 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0470929 May 24 10:11:37.032336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0be8abbb-a359-4570-8b40-d651020aac95 req-9fda38cf-068b-4e9c-a974-fbe6af4e161c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:37.033977 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0be8abbb-a359-4570-8b40-d651020aac95 req-9fda38cf-068b-4e9c-a974-fbe6af4e161c tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.95 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.2320812 May 24 10:11:37.268776 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:37.289426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Deleting port b5308ea7-0517-43e4-830b-7fd131b3a9c7 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:37.289426 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:37.444158 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f39ed61e-3850-416f-bd78-fc77a87b35c8 req-6b2e3ecf-da8c-4318-b156-9ba5bd250496 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:11:37.448159 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f39ed61e-3850-416f-bd78-fc77a87b35c8 req-6b2e3ecf-da8c-4318-b156-9ba5bd250496 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 945 time: 0.1653254 May 24 10:11:37.518738 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f39ed61e-3850-416f-bd78-fc77a87b35c8 req-546c5346-e22a-4907-a32a-7ac7ebb15236 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/security-groups?id=821159f9-8a27-4e94-8508-812d7562ee2c&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0639319 May 24 10:11:37.539040 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] For port b5308ea7-0517-43e4-830b-7fd131b3a9c7, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=b5308ea7-0517-43e4-830b-7fd131b3a9c7,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:37.539458 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:37.797583 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Calling delete_port for b5308ea7-0517-43e4-830b-7fd131b3a9c7 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:38.068706 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:38.076598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-3ca732f1-55d1-45e9-a52b-4823b34e2181 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:38.133358 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event deleted for resources: {'Port': ['ID=b5308ea7-0517-43e4-830b-7fd131b3a9c7,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:38.178937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.178937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.178937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.178937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.178937 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.179704 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.179704 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.179999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.180141 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.180569 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.180923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.181385 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:38.190122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:38.233027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] No DVR routers for this DVR port b5308ea7-0517-43e4-830b-7fd131b3a9c7 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:11:38.236598 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:38.242361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-deleted', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:11:38.344576 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0be8abbb-a359-4570-8b40-d651020aac95 req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.95 "DELETE /v2.0/ports/b5308ea7-0517-43e4-830b-7fd131b3a9c7 HTTP/1.1" status: 204 len: 173 time: 1.3037944 May 24 10:11:38.392846 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0eb2af08-e089-403d-8235-25b4d4ab656d', 'name': 'network-vif-deleted', 'tag': 'b5308ea7-0517-43e4-830b-7fd131b3a9c7', 'status': 'completed', 'code': 200} May 24 10:11:38.739713 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-aceb4d7b-7e26-4c51-92c4-4aee2197d21d req-38ea3463-b0eb-4f6b-91bd-876ae6ee6fce tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.0488589 May 24 10:11:38.828787 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:39.924233 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-faa43e46-ad2c-4635-aff7-7f9bea0f4b19 req-a88101a2-531f-4c6c-af19-87dae7440ba7 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Synchronizing usage tracker for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:11:40.034102 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-faa43e46-ad2c-4635-aff7-7f9bea0f4b19 req-a88101a2-531f-4c6c-af19-87dae7440ba7 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Unset dirty status for tenant:ce96e2adbf10489aae7640c84f398b29 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:11:40.035472 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-faa43e46-ad2c-4635-aff7-7f9bea0f4b19 req-a88101a2-531f-4c6c-af19-87dae7440ba7 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "GET /v2.0/ports?device_id=0eb2af08-e089-403d-8235-25b4d4ab656d HTTP/1.1" status: 200 len: 210 time: 0.1683748 May 24 10:11:41.159918 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33762) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:41.203779 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:41.254377 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: b5308ea7-0517-43e4-830b-7fd131b3a9c7 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:41.255734 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device b5308ea7-0517-43e4-830b-7fd131b3a9c7 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:41.287006 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:41.333453 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:41.519950 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:41.524970 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c691f1b9-9c78-48cb-8211-656b6e5414dd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:41.543296 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=821159f9-8a27-4e94-8508-812d7562ee2c,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:41.552666 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.553767 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.554140 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.554533 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.554914 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.555310 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.555687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.556062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.556438 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.556814 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.557244 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.557596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:41.561147 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:41.645365 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:41.647509 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] 10.222.0.67 "DELETE /v2.0/security-groups/821159f9-8a27-4e94-8508-812d7562ee2c HTTP/1.1" status: 204 len: 168 time: 0.4858856 May 24 10:11:41.777725 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33770) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:42.790170 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:43.324390 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Deleting port c366b56b-4d33-4772-8a44-91c29911caf1 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:43.324934 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:43.557275 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] For port c366b56b-4d33-4772-8a44-91c29911caf1, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c366b56b-4d33-4772-8a44-91c29911caf1,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:43.557628 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:43.704417 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Calling delete_port for c366b56b-4d33-4772-8a44-91c29911caf1 owned by network:router_interface {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:43.994035 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:43.998097 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-954f323e-7c27-401b-a21c-8f993dfea18c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:44.065502 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Pushing event deleted for resources: {'Port': ['ID=c366b56b-4d33-4772-8a44-91c29911caf1,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:44.073217 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:44.435523 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.005105 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Router 91d55ed4-1190-40a0-83e1-ece622fa9c2e has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:11:45.157958 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33802) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:45.229824 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:11:45.242712 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Sync routers for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:11:45.257523 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:45.258178 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:45.314405 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33808) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:45.314691 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-13c50943-a6e4-4565-8df2-211b71e43b7d admin admin] 10.222.0.67 "PUT /v2.0/routers/91d55ed4-1190-40a0-83e1-ece622fa9c2e/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.5353446 May 24 10:11:45.378284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Notify callbacks [] for security_group, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.420766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:11:45.428991 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Usage tracker for resource:security_group_rule and tenant:24ef9480cbd7427cb29c3364c0e024e9 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:11:45.433808 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:11:45.442603 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:11:45.454574 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:45.561687 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks [] for router, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.773026 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.786100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234'}], 'enable_snat': True}, 'gw_port_id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:11:43Z', 'revision_number': 5, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port': {'id': '34bf5469-5bdd-4b70-b23f-5015d71654e9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:06:3c:2b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 476, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.234', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:00Z', 'updated_at': '2021-05-24T10:06:07Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:45.791506 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Checking router: 91d55ed4-1190-40a0-83e1-ece622fa9c2e for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:11:45.801898 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Routers {'91d55ed4-1190-40a0-83e1-ece622fa9c2e'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:45.802505 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:45.891267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.934463 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Deleting port 34bf5469-5bdd-4b70-b23f-5015d71654e9 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:45.934797 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:45.969768 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] Set router 91d55ed4-1190-40a0-83e1-ece622fa9c2e gateway port host: None {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:46.016122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.027910 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0f544fc1-9ee6-48ae-b43f-ebfdfe3cc408 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:46.085505 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:11:45Z', 'revision_number': 6, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:46.090080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9d7de777-0e8b-4b0a-9a2c-3a12a667696c None None] The sync data for ids ['91d55ed4-1190-40a0-83e1-ece622fa9c2e'] in dv-cinder-791239-2-iscsi is: [{'id': '91d55ed4-1190-40a0-83e1-ece622fa9c2e', 'name': 'tempest-TestShelveInstance-1087220685-router', 'tenant_id': 'ce96e2adbf10489aae7640c84f398b29', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:05:59Z', 'updated_at': '2021-05-24T10:11:45Z', 'revision_number': 6, 'project_id': 'ce96e2adbf10489aae7640c84f398b29', 'gw_port_host': None}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:11:46.095550 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=24ef9480cbd7427cb29c3364c0e024e9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9362190 May 24 10:11:46.101300 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33812) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:46.111803 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=719ca57a-a725-4ed6-b1d3-2dfb00de367e,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:46.120156 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] For port 34bf5469-5bdd-4b70-b23f-5015d71654e9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=34bf5469-5bdd-4b70-b23f-5015d71654e9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:46.121551 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.197798 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.233373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.301433 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.273s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:46.368101 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Calling delete_port for 34bf5469-5bdd-4b70-b23f-5015d71654e9 owned by network:router_gateway {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:46.382369 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.388451 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-da36dc3e-d27a-4d51-a8bb-c77db6380b53 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:46.402033 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=719ca57a-a725-4ed6-b1d3-2dfb00de367e,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:46.404842 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:46.415518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.417377 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-32232c94-ceeb-423c-814d-da099c362eff admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/719ca57a-a725-4ed6-b1d3-2dfb00de367e HTTP/1.1" status: 204 len: 168 time: 0.3137603 May 24 10:11:46.628736 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:46.633486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:46.684221 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Pushing event deleted for resources: {'Port': ['ID=34bf5469-5bdd-4b70-b23f-5015d71654e9,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:46.687386 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:46.878066 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.016976 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c366b56b-4d33-4772-8a44-91c29911caf1 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:47.078583 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: c366b56b-4d33-4772-8a44-91c29911caf1 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:47.079362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c366b56b-4d33-4772-8a44-91c29911caf1 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:47.080045 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 34bf5469-5bdd-4b70-b23f-5015d71654e9 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:47.108913 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.112963 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.128706 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 34bf5469-5bdd-4b70-b23f-5015d71654e9 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:47.129153 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 34bf5469-5bdd-4b70-b23f-5015d71654e9 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:47.234292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.235168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 91d55ed4-1190-40a0-83e1-ece622fa9c2e {{(pid=109113) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:11:47.265626 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Notify callbacks [] for router, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.272692 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] 10.222.0.67 "DELETE /v2.0/routers/91d55ed4-1190-40a0-83e1-ece622fa9c2e HTTP/1.1" status: 204 len: 168 time: 1.9517624 May 24 10:11:47.272889 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33828) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:47.344032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Deleting subnet 0f050b21-5ec6-4623-a5c0-4e203399a27b {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:11:47.348109 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.418650 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.527026 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.916310 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:47.925448 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8bdfadc5-ec1f-4ec4-94be-2c0a7704396f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:47.994856 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Pushing event deleted for resources: {'Subnet': ['ID=0f050b21-5ec6-4623-a5c0-4e203399a27b,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:47.998517 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.005276 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4f99abba-4204-4002-82b5-ce29901e9630 None None] Network 9a3ab75c-0613-43fc-b406-9209bc3e8724 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:11:48.005537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:48.012920 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-51e7043d-31bf-495d-b8db-bdda72d3628f admin admin] 10.222.0.67 "DELETE /v2.0/subnets/0f050b21-5ec6-4623-a5c0-4e203399a27b HTTP/1.1" status: 204 len: 168 time: 0.7369399 May 24 10:11:48.013476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33844) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:48.173602 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.286561 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Deleting port 7ade3842-cd45-419d-9045-8626ba1fdbe0 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:48.288867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.374366 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] DHCP port deletion for 9a3ab75c-0613-43fc-b406-9209bc3e8724 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:11:48.446566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] For port 7ade3842-cd45-419d-9045-8626ba1fdbe0, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=7ade3842-cd45-419d-9045-8626ba1fdbe0,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:48.447029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.459423 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Deleting port 7ade3842-cd45-419d-9045-8626ba1fdbe0 {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:48.459985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.576642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] For port 7ade3842-cd45-419d-9045-8626ba1fdbe0, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=7ade3842-cd45-419d-9045-8626ba1fdbe0,segment=NetworkSegment(c69a7001-1e8e-470a-bfb0-2489247a3dd6),segment_id=c69a7001-1e8e-470a-bfb0-2489247a3dd6)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:48.576996 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:48.584167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Calling delete_port for 7ade3842-cd45-419d-9045-8626ba1fdbe0 owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:48.708496 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Calling delete_port for 7ade3842-cd45-419d-9045-8626ba1fdbe0 owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:48.728355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 7ade3842-cd45-419d-9045-8626ba1fdbe0 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:48.881371 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.286132 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.293380 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:49.352347 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.415810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:49.454829 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Pushing event deleted for resources: {'Port': ['ID=7ade3842-cd45-419d-9045-8626ba1fdbe0,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:49.470100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] No DVR routers for this DVR port 7ade3842-cd45-419d-9045-8626ba1fdbe0 on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:11:49.474080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.474407 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] No DVR routers for this DVR port 7ade3842-cd45-419d-9045-8626ba1fdbe0 on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:11:49.474680 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.475247 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.475855 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.476436 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.476880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Pushing event deleted for resources: {'Port': ['ID=7ade3842-cd45-419d-9045-8626ba1fdbe0,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:49.477038 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.477568 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.478767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.479497 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.480085 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.480649 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.481248 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:11:49.487500 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-a500c91f-c580-40d4-9eba-ca6738d7f2ee None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:49.505568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.212s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:49.612019 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.627881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['9a3ab75c-0613-43fc-b406-9209bc3e8724']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:49.646342 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'c69a7001-1e8e-470a-bfb0-2489247a3dd6') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:49.646918 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'c69a7001-1e8e-470a-bfb0-2489247a3dd6') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:49.662840 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.681440 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.689991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Releasing vlan 81 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:11:49.690684 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:11:49.716757 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:11:50.182444 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:50.187020 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-e3226215-d58e-414e-9759-25d88f4e366b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:50.202863 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:50.218218 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Pushing event deleted for resources: {'Network': ['ID=9a3ab75c-0613-43fc-b406-9209bc3e8724,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:50.223131 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:50.309007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:50.311795 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] 10.222.0.67 "DELETE /v2.0/networks/9a3ab75c-0613-43fc-b406-9209bc3e8724 HTTP/1.1" status: 204 len: 168 time: 2.2949300 May 24 10:11:50.313948 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33850) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:50.502978 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:11:50.956612 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:51.449249 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Deleting port c3854b9c-1314-4484-9d55-15d804931dda {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:51.450104 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:51.729468 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] For port c3854b9c-1314-4484-9d55-15d804931dda, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=c3854b9c-1314-4484-9d55-15d804931dda,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:51.729960 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:51.905372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Calling delete_port for c3854b9c-1314-4484-9d55-15d804931dda owned by network:router_interface {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:52.188468 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:52.193019 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-9279a2b7-32fb-40ef-82ef-3cc3d5a2e744 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:52.251073 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Pushing event deleted for resources: {'Port': ['ID=c3854b9c-1314-4484-9d55-15d804931dda,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:52.257287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:52.655181 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:53.126527 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Router 0d750bba-941f-408f-9a1d-724b07fb0aae has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:11:53.333301 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:11:53.343909 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] 10.222.0.67 "PUT /v2.0/routers/0d750bba-941f-408f-9a1d-724b07fb0aae/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.0287149 May 24 10:11:53.346334 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Sync routers for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:11:53.348120 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33868) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:53.370924 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:53.378736 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:53.678343 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:53.851113 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks [] for router, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:54.070097 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:11:52Z', 'revision_number': 5, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:54.074891 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Checking router: 0d750bba-941f-408f-9a1d-724b07fb0aae for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:11:54.086585 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Routers {'0d750bba-941f-408f-9a1d-724b07fb0aae'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:11:54.087257 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:11:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['0d750bba-941f-408f-9a1d-724b07fb0aae'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:54.159388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:54.372367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Deleting port e2803d5b-53e2-4d18-9296-07a8b49dd7c9 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:54.372888 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:54.428023 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] Set router 0d750bba-941f-408f-9a1d-724b07fb0aae gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:11:54.533949 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] For port e2803d5b-53e2-4d18-9296-07a8b49dd7c9, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:54.537509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:54.693718 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Calling delete_port for e2803d5b-53e2-4d18-9296-07a8b49dd7c9 owned by network:router_gateway {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:54.779398 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:11:52Z', 'revision_number': 5, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:54.784289 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2a07c2f2-80d0-4cd4-8369-d4a7492fdb3a None None] The sync data for ids ['0d750bba-941f-408f-9a1d-724b07fb0aae'] in dv-cinder-791239-2-iscsi is: [{'id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'name': 'tempest-TestShelveInstance-1246651160-router', 'tenant_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252'}], 'enable_snat': True}, 'gw_port_id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:06:12Z', 'updated_at': '2021-05-24T10:11:52Z', 'revision_number': 5, 'project_id': '11d14c1ea7714a25b90a7b8ea78444f3', 'gw_port': {'id': 'e2803d5b-53e2-4d18-9296-07a8b49dd7c9', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:b3:6b:71', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0d750bba-941f-408f-9a1d-724b07fb0aae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 499, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.252', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:06:13Z', 'updated_at': '2021-05-24T10:06:20Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:11:54.977000 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:54.983062 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8296f0cc-4c9e-4c2d-b5dc-ac5e1daaf38a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:55.032507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Pushing event deleted for resources: {'Port': ['ID=e2803d5b-53e2-4d18-9296-07a8b49dd7c9,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:55.102772 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:55.216167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.551355 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.555211 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.742458 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.743259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 0d750bba-941f-408f-9a1d-724b07fb0aae {{(pid=109112) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:11:55.781578 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.781578 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-7bbcd706-6796-4b1a-b30e-c13de4d42134 admin admin] 10.222.0.67 "DELETE /v2.0/routers/0d750bba-941f-408f-9a1d-724b07fb0aae HTTP/1.1" status: 204 len: 168 time: 2.4154985 May 24 10:11:55.781578 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33884) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:55.841013 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Deleting subnet fa4071c3-553b-41c3-bee4-024bec5e4c11 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:11:55.844583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.903814 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:55.985707 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:56.468475 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:56.475931 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d32684f0-57d4-4013-aefe-868b56edb900 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:56.534488 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=fa4071c3-553b-41c3-bee4-024bec5e4c11,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:56.540071 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:56.546272 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:56.551055 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-000686de-cebb-4bb0-b7af-ec0478792466 None None] Network 06601b3d-f3ef-4ad5-b8fc-080d71f88279 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:11:56.553851 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/fa4071c3-553b-41c3-bee4-024bec5e4c11 HTTP/1.1" status: 204 len: 168 time: 0.7840157 May 24 10:11:56.560155 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33894) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:56.763426 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:56.805021 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e2803d5b-53e2-4d18-9296-07a8b49dd7c9 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:56.913988 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Deleting port 8b325555-7840-441d-aa3f-d98efeaa4c4f {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:56.914966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:56.924638 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: e2803d5b-53e2-4d18-9296-07a8b49dd7c9 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:56.925611 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device e2803d5b-53e2-4d18-9296-07a8b49dd7c9 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:56.926600 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c3854b9c-1314-4484-9d55-15d804931dda no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:57.013407 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: c3854b9c-1314-4484-9d55-15d804931dda {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:57.013893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device c3854b9c-1314-4484-9d55-15d804931dda not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:57.101181 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] For port 8b325555-7840-441d-aa3f-d98efeaa4c4f, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8b325555-7840-441d-aa3f-d98efeaa4c4f,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:57.102069 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.164414 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7684033f-2177-4574-b171-f20a96a23905 None None] DHCP port deletion for 06601b3d-f3ef-4ad5-b8fc-080d71f88279 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:11:57.292780 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Deleting port 8b325555-7840-441d-aa3f-d98efeaa4c4f {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:11:57.293409 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.322375 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Calling delete_port for 8b325555-7840-441d-aa3f-d98efeaa4c4f owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:57.464288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-7684033f-2177-4574-b171-f20a96a23905 None None] For port 8b325555-7840-441d-aa3f-d98efeaa4c4f, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=8b325555-7840-441d-aa3f-d98efeaa4c4f,segment=NetworkSegment(feab763c-9341-411a-b433-e4628ba3e968),segment_id=feab763c-9341-411a-b433-e4628ba3e968)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:11:57.464288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.479831 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.489712 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:57.587313 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Pushing event deleted for resources: {'Port': ['ID=8b325555-7840-441d-aa3f-d98efeaa4c4f,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:57.593399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:57.598418 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] No DVR routers for this DVR port 8b325555-7840-441d-aa3f-d98efeaa4c4f on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:11:57.644115 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Calling delete_port for 8b325555-7840-441d-aa3f-d98efeaa4c4f owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:11:57.769543 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.784980 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['06601b3d-f3ef-4ad5-b8fc-080d71f88279']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:57.790758 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.800032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'feab763c-9341-411a-b433-e4628ba3e968') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:57.800383 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'feab763c-9341-411a-b433-e4628ba3e968') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:11:57.801164 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:57.813255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.839864 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.847946 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Releasing vlan 92 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:11:57.848498 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:11:57.868395 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:11:57.876137 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Pushing event deleted for resources: {'Port': ['ID=8b325555-7840-441d-aa3f-d98efeaa4c4f,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:57.881571 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:57.885293 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7684033f-2177-4574-b171-f20a96a23905 None None] No DVR routers for this DVR port 8b325555-7840-441d-aa3f-d98efeaa4c4f on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:11:58.016401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:58.025728 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:58.025728 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:58.058467 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Pushing event deleted for resources: {'Network': ['ID=06601b3d-f3ef-4ad5-b8fc-080d71f88279,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:58.061763 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:58.066879 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] 10.222.0.67 "DELETE /v2.0/networks/06601b3d-f3ef-4ad5-b8fc-080d71f88279 HTTP/1.1" status: 204 len: 168 time: 1.5049069 May 24 10:11:58.692075 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33930) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:58.742826 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8b325555-7840-441d-aa3f-d98efeaa4c4f no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:11:58.785275 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d4734ef4-9b35-4aa9-9e5e-28b6bc8edd1f admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=ce96e2adbf10489aae7640c84f398b29&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0913553 May 24 10:11:58.788339 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 33934) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:11:58.809908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 8b325555-7840-441d-aa3f-d98efeaa4c4f {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:11:58.810526 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 8b325555-7840-441d-aa3f-d98efeaa4c4f not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:11:58.874816 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:58.907203 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:59.062467 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:59.070789 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c3936c63-d710-43dd-8dd2-763f5b8436a5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:11:59.090401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=11b49c10-dd6e-4ac3-84b3-9623e9146b6d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:11:59.094783 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:11:59.098411 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:11:59.100862 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-fc35f288-0d62-4c9a-a5e2-dd7bb3747276 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/11b49c10-dd6e-4ac3-84b3-9623e9146b6d HTTP/1.1" status: 204 len: 168 time: 0.3104596 May 24 10:11:59.977080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33964) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:00.046534 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-3e399ec4-b35a-4d4d-9d3c-6e4881a751b2 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=11d14c1ea7714a25b90a7b8ea78444f3&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0668113 May 24 10:12:00.049682 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 33970) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:00.135520 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:00.176724 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:00.354343 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:00.363444 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-52be618e-3378-44f3-9750-80631c6311ab admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:00.385403 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ab9e58e0-9c36-4c8f-8dc8-3488ab897f48,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:00.391829 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:00.395449 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:00.396473 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-e568c617-5bc0-4e11-bd01-72a69215890f admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/ab9e58e0-9c36-4c8f-8dc8-3488ab897f48 HTTP/1.1" status: 204 len: 168 time: 0.3448145 May 24 10:12:03.376396 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-548c424b-fae5-43ba-8ab5-1ed48269ae37 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.216883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34002) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:05.367860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.397642 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:05.402248 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Usage tracker for resource:security_group_rule and tenant:c6ef2fedd4844913ab8a14901b560ced is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:05.405548 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:05.412355 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:05.507077 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.576797 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.588328 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-37cd1ebf-1721-4536-b0da-3641fdab34e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:05.634410 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=c6ef2fedd4844913ab8a14901b560ced&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4160783 May 24 10:12:05.635541 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=05a4cc5e-3a77-4993-957b-e0dd15a921d0,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:05.637672 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34008) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:05.639894 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4c35abed-4293-4e47-9676-d58c7a860857 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:05.723931 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.773102 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.812907 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ed7372a8-da5e-4f40-acc3-f5b6a73a6b49 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.959731 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.960088 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.965097 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1ae2adc2-fb04-42c2-aec5-814f412c78d1 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:05.977787 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=05a4cc5e-3a77-4993-957b-e0dd15a921d0,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:05.982958 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:05.988318 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:05.990008 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-90ab90cf-a9a1-4ea4-a7e9-300b126da1da admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/05a4cc5e-3a77-4993-957b-e0dd15a921d0 HTTP/1.1" status: 204 len: 168 time: 0.3507824 May 24 10:12:07.262543 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.295402 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34032) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:07.421667 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-1417893617-network', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:07.426972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:07.432379 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Usage tracker for resource:network and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:07.437282 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:07.444928 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:07.518833 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Made reservation on behalf of 2cc9bd0e4be64b0e9add9a9f9e8bbe52 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:12:07.521361 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.538581 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.566939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:07.574679 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:07.582033 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:07.593856 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:07.719174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.810518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.813817 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-189be8d6-161c-4264-a31b-b2b441187580 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:07.849434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 82} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:12:07.855571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1a5958ce-3459-45c8-8025-29ec96e20aae,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:07.861398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:07.867302 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:07.867670 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Added segment 190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99 of type vlan for network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 May 24 10:12:07.912446 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:08.056171 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:08.057823 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:08.058773 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:08.063372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-87dcf526-772d-4a5e-922f-923e8c5e1277 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:08.189709 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Pushing event updated for resources: {'Network': ['ID=3670d884-4bc4-4d4e-8dbb-1165ed224ea7,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:08.192807 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:08.204254 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:12:08.268493 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 0.9714963 May 24 10:12:08.272029 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34042) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:08.278927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Request body: {'subnet': {'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-1417893617-subnet', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:08.285388 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:08.291476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Usage tracker for resource:subnet and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:08.295741 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:08.304947 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:08.376190 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Made reservation on behalf of 2cc9bd0e4be64b0e9add9a9f9e8bbe52 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:12:08.379189 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:08.822423 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.037552 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.041964 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8fa5f2a7-b55c-4bb8-a58d-474f9bdfa082 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:09.118764 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Pushing event updated for resources: {'Subnet': ['ID=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:09.123428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:09.133714 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:09.134083 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Network hostable DHCP agents. Network: 3670d884-4bc4-4d4e-8dbb-1165ed224ea7, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:12:09.233163 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:12:09.365555 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 3670d884-4bc4-4d4e-8dbb-1165ed224ea7) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:09.375450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:09.376119 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.377281 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-db67ed9e-5dd3-4436-990a-55e41a8cb5d4 None None] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:12:09.520108 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f256d81f-4974-42b7-a82b-ad2fffe6040c admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 848 time: 1.2464173 May 24 10:12:09.523883 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34046) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:09.530080 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:09.538557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:09.545780 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Usage tracker for resource:router and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:09.549391 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:09.557566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:09.637441 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:12:09.638373 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.707558 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.709546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.732782 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:09.820986 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Allocated IP 172.20.0.2 (3670d884-4bc4-4d4e-8dbb-1165ed224ea7/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51/bb5b081c-fde7-4536-9096-eefa2bf2806a) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:10.055833 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.067191 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Transition to ACTIVE for port object bb5b081c-fde7-4536-9096-eefa2bf2806a will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:10.090098 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Transition to ACTIVE for port object bb5b081c-fde7-4536-9096-eefa2bf2806a will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:10.204950 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.212040 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:10.244385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.294325 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:10.297623 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:10.301752 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:10.308823 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Pushing event updated for resources: {'Port': ['ID=bb5b081c-fde7-4536-9096-eefa2bf2806a,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:10.313336 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:10.407802 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Attempting to bind port bb5b081c-fde7-4536-9096-eefa2bf2806a on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:10.408179 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Attempting to bind port bb5b081c-fde7-4536-9096-eefa2bf2806a by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:10.408573 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Attempting to bind port bb5b081c-fde7-4536-9096-eefa2bf2806a on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:10.415877 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:10.416265 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:10.416692 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.417150 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Bound using segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:10.417654 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Bound port: bb5b081c-fde7-4536-9096-eefa2bf2806a, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:10.418068 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.518053 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] For port bb5b081c-fde7-4536-9096-eefa2bf2806a, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:10.521846 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] For port bb5b081c-fde7-4536-9096-eefa2bf2806a, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=bb5b081c-fde7-4536-9096-eefa2bf2806a,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:10.616620 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.621555 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:10.727941 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Pushing event updated for resources: {'Port': ['ID=bb5b081c-fde7-4536-9096-eefa2bf2806a,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:10.734078 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:10.767149 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:10.821369 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:11.028259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Allocated IP 172.27.0.138 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/9a20f529-25b1-4304-a42c-29516af43de2) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:11.244577 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:11.462258 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:11.466463 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-af562234-0324-4f2f-86d2-210face6bb3d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:11.540706 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:11.544428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:11.547845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:11.597455 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Pushing event updated for resources: {'Port': ['ID=9a20f529-25b1-4304-a42c-29516af43de2,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:11.601575 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:11.912412 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-830fbc84-8dab-40e8-b90b-a2539e4834b7 None None] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:12:12.041272 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-9645d69c-53d5-4003-9563-675daa4f2f4c None None] Provisioning for port bb5b081c-fde7-4536-9096-eefa2bf2806a completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:12.416638 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:12.764355 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device bb5b081c-fde7-4536-9096-eefa2bf2806a up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:12.901721 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port bb5b081c-fde7-4536-9096-eefa2bf2806a completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:12.907190 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port bb5b081c-fde7-4536-9096-eefa2bf2806a triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:12.907572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:13.018739 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:13.037857 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:13.119449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port bb5b081c-fde7-4536-9096-eefa2bf2806a, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=bb5b081c-fde7-4536-9096-eefa2bf2806a,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:13.210506 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:13.215438 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:13.289930 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=bb5b081c-fde7-4536-9096-eefa2bf2806a,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:13.293668 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:13.490583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Router 6db26429-7a8b-455b-8586-398e35a18a57 is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:12:13.708780 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:12:13.720362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:12:13.729503 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:13.730146 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:13.799374 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:13.801414 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 924 time: 4.2761703 May 24 10:12:13.805116 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34050) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:13.965149 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:14.401843 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:12Z', 'revision_number': 3, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:11Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:14.408992 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:12:14.409575 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Updating router 6db26429-7a8b-455b-8586-398e35a18a57 port 9a20f529-25b1-4304-a42c-29516af43de2 binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:12:14.508166 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:14.756900 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device bb5b081c-fde7-4536-9096-eefa2bf2806a up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:14.813831 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.026669 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port bb5b081c-fde7-4536-9096-eefa2bf2806a triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:15.027056 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.079414 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Allocated IP 172.20.0.1 (3670d884-4bc4-4d4e-8dbb-1165ed224ea7/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51/b64ed3ba-3823-4d02-9656-ce21e0789e74) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:15.242812 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.253939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.263046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Transition to ACTIVE for port object 9a20f529-25b1-4304-a42c-29516af43de2 will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:15.272583 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.provisioning_blocks [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Transition to ACTIVE for port object b64ed3ba-3823-4d02-9656-ce21e0789e74 will not be triggered until provisioned by entity DHCP. {{(pid=109112) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:15.415619 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.419773 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-5e787b8f-3c74-457a-8112-cedf1d3b090f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:15.428545 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.437007 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ab410c54-460f-4df9-9c2f-5c2ab3da90ec tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:15.506944 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Attempting to bind port 9a20f529-25b1-4304-a42c-29516af43de2 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:15.507329 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Attempting to bind port 9a20f529-25b1-4304-a42c-29516af43de2 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:15.507704 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Attempting to bind port 9a20f529-25b1-4304-a42c-29516af43de2 on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:15.514107 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-202a1a3e-12c3-42a9-8f6b-23c78fc623ad None None] Pushing event updated for resources: {'Port': ['ID=9a20f529-25b1-4304-a42c-29516af43de2,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:15.517477 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-202a1a3e-12c3-42a9-8f6b-23c78fc623ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:15.520679 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:15.521189 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:15.521518 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.521918 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:15.522565 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Bound port: 9a20f529-25b1-4304-a42c-29516af43de2, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:15.522971 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.560237 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:15.565889 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:15.572707 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:15.623694 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] For port 9a20f529-25b1-4304-a42c-29516af43de2, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:15.629571 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] For port 9a20f529-25b1-4304-a42c-29516af43de2, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=9a20f529-25b1-4304-a42c-29516af43de2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:15.668537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Pushing event updated for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:15.674349 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.237s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:15.709430 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.759433 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:15.765179 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7ec21690-3a8c-4ba6-84c5-aa5bb9c617c1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:15.875360 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:15.876069 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:15.952296 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-202a1a3e-12c3-42a9-8f6b-23c78fc623ad None None] Pushing event updated for resources: {'Port': ['ID=9a20f529-25b1-4304-a42c-29516af43de2,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:15.955537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-202a1a3e-12c3-42a9-8f6b-23c78fc623ad None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:16.098237 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-0811e565-eb67-4572-ac4a-69680f00c506 None None] Provisioning for port b64ed3ba-3823-4d02-9656-ce21e0789e74 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:16.133323 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-0811e565-eb67-4572-ac4a-69680f00c506 None None] Provisioning complete for port b64ed3ba-3823-4d02-9656-ce21e0789e74 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:16.133939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-0811e565-eb67-4572-ac4a-69680f00c506 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:16.156398 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:16.249401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:16.358486 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-0811e565-eb67-4572-ac4a-69680f00c506 None None] Port b64ed3ba-3823-4d02-9656-ce21e0789e74 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:12:16.490134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:16.559164 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:16.597610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f2222ff8-9342-4d38-a7cb-779e895940ef admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:16.600906 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:12Z', 'revision_number': 3, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:16.607936 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d5890060-e729-47f5-ad50-783d82d9108e None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:12Z', 'revision_number': 3, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:12:16.725910 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Pushing event updated for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:16.733343 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:16.738404 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:17.208928 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:17.211102 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:17.479899 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:12:17.496918 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6aae3457-9664-47da-81e5-d48396c50fc5 None None] Provisioning complete for port b64ed3ba-3823-4d02-9656-ce21e0789e74 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:17.497511 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6aae3457-9664-47da-81e5-d48396c50fc5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:17.576732 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-6aae3457-9664-47da-81e5-d48396c50fc5 None None] Port b64ed3ba-3823-4d02-9656-ce21e0789e74 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:12:17.698727 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:12:17.709127 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-8d8290b3-b31c-4e38-865a-d0a38efc283c admin admin] 10.222.0.67 "PUT /v2.0/routers/6db26429-7a8b-455b-8586-398e35a18a57/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.9027355 May 24 10:12:18.301735 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:12:18.317390 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:18.318489 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:18.521162 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:18.672211 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34068) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:18.682348 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-949955037-network', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:18.689179 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:18.696469 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Usage tracker for resource:network and tenant:0850b91daf724bcd93114c2b6acaf926 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:18.704134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:18.720053 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:18.797645 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 9a20f529-25b1-4304-a42c-29516af43de2 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:18.833510 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Made reservation on behalf of 0850b91daf724bcd93114c2b6acaf926 for: {'network': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:12:18.836343 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-3570195'] for network, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:18.858349 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:18.879415 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:18.885894 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Usage tracker for resource:security_group_rule and tenant:0850b91daf724bcd93114c2b6acaf926 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:18.890552 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:18.891808 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:16Z', 'revision_number': 2, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:18.897430 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:12:18.897851 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Updating router 6db26429-7a8b-455b-8586-398e35a18a57 port b64ed3ba-3823-4d02-9656-ce21e0789e74 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:12:18.900409 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:18.962433 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.010690 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port 9a20f529-25b1-4304-a42c-29516af43de2 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:19.019545 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 9a20f529-25b1-4304-a42c-29516af43de2 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:19.020862 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.025017 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.095416 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.100774 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-004a753a-dcb6-41b6-bff6-d681fa5f7189 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:19.148492 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 89} {{(pid=109112) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 10:12:19.151620 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=53727b75-01be-405f-a6ce-369272629f50,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:19.156346 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:19.171937 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.172610 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.segments_db [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Added segment 9726e60b-1b4d-46d9-9951-468c3217b9eb of type vlan for network 87e2b5b7-b682-411b-ad8f-70bb178250f4 May 24 10:12:19.214689 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8762070508638'] for network, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.253521 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.287337 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port 9a20f529-25b1-4304-a42c-29516af43de2, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=9a20f529-25b1-4304-a42c-29516af43de2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:19.324064 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031'] for network, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.325278 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:19.326049 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.333641 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:19.354249 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.375159 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Transition to ACTIVE for port object b64ed3ba-3823-4d02-9656-ce21e0789e74 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:19.387280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.390323 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:19.451609 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.458741 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Pushing event updated for resources: {'Network': ['ID=87e2b5b7-b682-411b-ad8f-70bb178250f4,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:19.460478 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:19.463860 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:19.474893 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 details requested by agent ovs-agent-dv-cinder-791239-2-iscsi with host dv-cinder-791239-2-iscsi {{(pid=109114) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 10:12:19.519788 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=9a20f529-25b1-4304-a42c-29516af43de2,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:19.526235 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] 10.222.0.67 "POST /v2.0/networks HTTP/1.1" status: 201 len: 883 time: 0.8477447 May 24 10:12:19.526530 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:19.527879 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34072) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:19.547999 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Request body: {'subnet': {'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-949955037-subnet', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'ip_version': 4}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:19.568021 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:19.568229 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Usage tracker for resource:subnet and tenant:0850b91daf724bcd93114c2b6acaf926 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:19.577404 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:19.591553 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:19.674763 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Pushing event updated for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:19.683172 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Made reservation on behalf of 0850b91daf724bcd93114c2b6acaf926 for: {'subnet': 1} {{(pid=109112) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:12:19.686727 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1461640'] for subnet, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.740802 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.741303 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.741569 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.741900 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.742249 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.742576 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.742896 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.743211 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.743531 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.743847 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.744159 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.744472 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109115) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:19.748581 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.288s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:19.767460 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Attempting to bind port b64ed3ba-3823-4d02-9656-ce21e0789e74 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:19.767983 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Attempting to bind port b64ed3ba-3823-4d02-9656-ce21e0789e74 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:19.768452 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Attempting to bind port b64ed3ba-3823-4d02-9656-ce21e0789e74 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:19.775223 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:19.775777 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:19.776290 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.776786 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Bound using segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:19.777411 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Bound port: b64ed3ba-3823-4d02-9656-ce21e0789e74, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:19.777884 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.874325 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] For port b64ed3ba-3823-4d02-9656-ce21e0789e74, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:19.878725 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] For port b64ed3ba-3823-4d02-9656-ce21e0789e74, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b64ed3ba-3823-4d02-9656-ce21e0789e74,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:19.965025 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:19.969298 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:20.071494 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:20.072151 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:20.121090 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Pushing event updated for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=4']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:20.124848 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-87e0a2cd-f7eb-4599-940b-4ee290d87022 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:20.221458 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-127969874', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1461662'] for subnet, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.226289 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:20.250625 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.312102 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Pushing event updated for resources: {'Subnet': ['ID=8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:20.313072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:20.320744 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:20.321368 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Checking segment: {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109112) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:20.321727 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Network hostable DHCP agents. Network: 87e2b5b7-b682-411b-ad8f-70bb178250f4, hostable agents: ['cc61fad8-5fd3-4da5-857c-bfda623021a0'], hosted agents: [] {{(pid=109112) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 10:12:20.444813 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 is scheduled to be hosted by DHCP agent cc61fad8-5fd3-4da5-857c-bfda623021a0 with binding_index 1 {{(pid=109112) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 10:12:20.564165 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-791239-2-iscsi, 87e2b5b7-b682-411b-ad8f-70bb178250f4) {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:20.577947 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-55d078bb-04d4-409b-bba5-b7d1da3cbe23 None None] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:12:20.581751 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:20.582724 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.678536 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:20.686096 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-198e2f0f-7122-443b-b087-64fd612eab7e None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:12:20.714852 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-204d81a5-156b-4fd4-a05b-9c019897e0ef admin admin] 10.222.0.67 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 1.1853600 May 24 10:12:20.752115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34076) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:20.752115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:20.752115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:20.752115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Usage tracker for resource:router and tenant:0850b91daf724bcd93114c2b6acaf926 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:20.752115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:20.768214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:20.837556 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 9a20f529-25b1-4304-a42c-29516af43de2 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:20.884899 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-423171', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4107'] for router, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.886880 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-422403', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-425436', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-3756081', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4210'] for router, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.911733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:20.959035 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpc7d75e06-cff1-553a-8747-fe85b4f162f8-87e2b5b7-b682-411b-ad8f-70bb178250f4', 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'fixed_ips': [{'subnet_id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb'}]}} from dv-cinder-791239-2-iscsi. {{(pid=109114) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 10:12:20.960135 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.029262 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port 9a20f529-25b1-4304-a42c-29516af43de2 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:21.029650 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.241988 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.db_base_plugin_common [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Allocated IP 172.20.0.2 (87e2b5b7-b682-411b-ad8f-70bb178250f4/8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb/f97a5e38-df31-42ff-b538-474375dde42d) {{(pid=109114) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:21.543927 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks [] for port, precommit_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.556601 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Transition to ACTIVE for port object f97a5e38-df31-42ff-b538-474375dde42d will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:21.584877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-425465', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-423217', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4196'] for router, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.586210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Transition to ACTIVE for port object f97a5e38-df31-42ff-b538-474375dde42d will not be triggered until provisioned by entity DHCP. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:21.667198 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380'] for port, after_create {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.672365 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:21.757341 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 is already hosted by enough agents. {{(pid=109114) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:21.760537 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.scheduler.base_scheduler [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Resources bound (agent type, host, resource id): {{(pid=109114) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:21.763490 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Received port after_create {{(pid=109114) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:21.824895 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Pushing event updated for resources: {'Port': ['ID=f97a5e38-df31-42ff-b538-474375dde42d,revision_number=1']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:21.886750 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.214s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:21.924977 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Attempting to bind port f97a5e38-df31-42ff-b538-474375dde42d on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:21.925662 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Attempting to bind port f97a5e38-df31-42ff-b538-474375dde42d by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:21.926908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Attempting to bind port f97a5e38-df31-42ff-b538-474375dde42d on network 87e2b5b7-b682-411b-ad8f-70bb178250f4 {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:21.937806 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:21.938442 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Checking segment: {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:21.939128 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:21.939822 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Bound using segment: {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:21.940677 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Bound port: f97a5e38-df31-42ff-b538-474375dde42d, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:21.941382 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.068576 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] For port f97a5e38-df31-42ff-b538-474375dde42d, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:22.073238 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] For port f97a5e38-df31-42ff-b538-474375dde42d, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=f97a5e38-df31-42ff-b538-474375dde42d,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:22.107678 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.140484 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.187362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.194951 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:22.309997 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Allocated IP 172.27.0.125 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:22.341281 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Pushing event updated for resources: {'Port': ['ID=f97a5e38-df31-42ff-b538-474375dde42d,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:22.346853 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:22.577578 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.774801 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:22.777758 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-530a4c66-4529-4bcb-bf66-06af2a96fa72 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:22.797682 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device f97a5e38-df31-42ff-b538-474375dde42d up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:22.869377 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:22.875226 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:22.883238 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:22.977602 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Pushing event updated for resources: {'Port': ['ID=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:22.984746 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:23.023607 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port f97a5e38-df31-42ff-b538-474375dde42d completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:23.030742 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b64ed3ba-3823-4d02-9656-ce21e0789e74 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:23.222504 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port b64ed3ba-3823-4d02-9656-ce21e0789e74 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:23.231674 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port b64ed3ba-3823-4d02-9656-ce21e0789e74 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:23.232142 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.375549 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.386616 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-560e8b27-8c19-4b04-b361-ed24ff12c102 None None] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:12:23.515527 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port b64ed3ba-3823-4d02-9656-ce21e0789e74, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b64ed3ba-3823-4d02-9656-ce21e0789e74,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:23.524056 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Provisioning for port f97a5e38-df31-42ff-b538-474375dde42d completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:23.533317 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Provisioning complete for port f97a5e38-df31-42ff-b538-474375dde42d triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:23.533786 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.609582 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.636029 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.641557 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-7684033f-2177-4574-b171-f20a96a23905 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:23.813457 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:23.828181 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.187s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:23.847163 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:23.898192 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] For port f97a5e38-df31-42ff-b538-474375dde42d, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=f97a5e38-df31-42ff-b538-474375dde42d,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:24.015245 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:24.017560 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:24.125469 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Pushing event updated for resources: {'Port': ['ID=f97a5e38-df31-42ff-b538-474375dde42d,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:24.133802 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-feb17e7f-0c78-4d69-844e-70f2bb6b7b30 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:24.238731 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-cee47a8c-8a3b-4e65-9c09-3c1f9a853efc None None] Provisioning complete for port f97a5e38-df31-42ff-b538-474375dde42d triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:24.239632 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-cee47a8c-8a3b-4e65-9c09-3c1f9a853efc None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:24.294544 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-423178', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-425402'] for router, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:24.616036 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad is scheduled to L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd with binding_index 1 {{(pid=109112) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 10:12:24.787680 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device f97a5e38-df31-42ff-b538-474375dde42d up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:24.847485 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:12:24.863438 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Sync routers for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:12:24.886426 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:24.887139 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:24.964180 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:24.966168 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] 10.222.0.67 "POST /v2.0/routers HTTP/1.1" status: 201 len: 923 time: 4.2474389 May 24 10:12:24.970708 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34080) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:24.987000 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port f97a5e38-df31-42ff-b538-474375dde42d triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:24.987394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:25.188372 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:25.455238 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b64ed3ba-3823-4d02-9656-ce21e0789e74 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:25.593215 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port b64ed3ba-3823-4d02-9656-ce21e0789e74 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:25.593603 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:25.670132 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 3, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:22Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:25.679767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Checking router: 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:12:25.679767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Updating router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd binding host dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:12:25.789336 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.165679 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.300952 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks [] for port, precommit_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.325042 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Transition to ACTIVE for port object eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd will not be triggered until provisioned by entity L2. {{(pid=109114) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:26.387023 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.393564 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:26.418018 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Allocated IP 172.20.0.1 (87e2b5b7-b682-411b-ad8f-70bb178250f4/8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb/d3ce81dc-f3b2-4725-90b6-77a7e878de97) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:26.513418 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Attempting to bind port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:26.514013 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Attempting to bind port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:26.514629 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Attempting to bind port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd on network 2211f181-5fdc-4580-9f8a-8610ff86d56a {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:26.532546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:26.533214 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Checking segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109114) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:26.533927 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.534849 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Bound using segment: {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'} {{(pid=109114) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:26.535669 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.managers [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Bound port: eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '7bdf7a39-c927-428c-ae01-d27b28cde973', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a'}}] {{(pid=109114) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:26.536822 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.580036 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-792ee7f4-d995-4c01-8fcd-d39ae9256a53 None None] Pushing event updated for resources: {'Port': ['ID=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,revision_number=2']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:26.599028 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-792ee7f4-d995-4c01-8fcd-d39ae9256a53 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.205s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:26.601480 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.620115 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Transition to ACTIVE for port object d3ce81dc-f3b2-4725-90b6-77a7e878de97 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:26.667625 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] For port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109114) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:26.675851 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] For port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:26.747386 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.751999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:26.781448 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:26.786934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:26.825552 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:26.828827 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:26.833310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:26.866120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Pushing event updated for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:26.873081 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:26.888377 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:26.889404 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:26.906292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-425163'] for router_interface, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:27.000504 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-792ee7f4-d995-4c01-8fcd-d39ae9256a53 None None] Pushing event updated for resources: {'Port': ['ID=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,revision_number=3']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:27.006149 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-792ee7f4-d995-4c01-8fcd-d39ae9256a53 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:27.195630 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:27.296679 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-1a61f548-51cd-43fe-ad10-2bd4f6c5198e None None] Provisioning for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:27.340230 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-1a61f548-51cd-43fe-ad10-2bd4f6c5198e None None] Provisioning complete for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:27.341398 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1a61f548-51cd-43fe-ad10-2bd4f6c5198e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:27.408470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:27.495212 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:12:27.555881 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-1a61f548-51cd-43fe-ad10-2bd4f6c5198e None None] Port d3ce81dc-f3b2-4725-90b6-77a7e878de97 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:12:27.685151 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 3, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:26Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:27.692425 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac27dfc0-a705-47a2-be84-0345dae055db None None] The sync data for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi is: [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 3, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:26Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:12:27.727414 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:27.861987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:27.910775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-65966675-7ff5-422d-8b7b-40eec66c21dd tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:28.048190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Pushing event updated for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:28.051594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:28.055950 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-3925101', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-425062'] for router_interface, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:28.248904 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:28.251146 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:28.591314 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-eada5397-4cc2-4015-b4a3-3870d24724f9 None None] Provisioning complete for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:28.592199 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-eada5397-4cc2-4015-b4a3-3870d24724f9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:28.654532 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:12:28.655410 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-eada5397-4cc2-4015-b4a3-3870d24724f9 None None] Port d3ce81dc-f3b2-4725-90b6-77a7e878de97 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:12:28.793463 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:28.929418 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:28.934900 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:28.935295 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:28.963019 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:12:28.980260 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] 10.222.0.67 "PUT /v2.0/routers/8f6d9311-60c1-4a8b-abc6-ac26dcd1daad/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.0076106 May 24 10:12:29.103704 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:29.125441 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:29.181597 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:29.184655 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-202a1a3e-12c3-42a9-8f6b-23c78fc623ad None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:29.309480 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:29.313121 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:29.552696 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Sync routers for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:12:29.568109 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:29.570354 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:29.846023 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:30.075840 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34090) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:30.081965 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-525824861', 'description': 'tempest-secgroup-smoke-525824861 description', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:30.088658 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:30.093556 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:30.099201 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:30.105949 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:30.193251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.216808 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:30.221508 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:30.225516 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:30.233200 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:30.252544 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:27Z', 'revision_number': 4, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'd3ce81dc-f3b2-4725-90b6-77a7e878de97', 'name': '', 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'mac_address': 'fa:16:3e:c4:5f:93', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_interface', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:26Z', 'updated_at': '2021-05-24T10:12:27Z', 'revision_number': 2, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'subnets': [{'id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:30.257693 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Checking router: 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:12:30.258050 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Updating router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad port d3ce81dc-f3b2-4725-90b6-77a7e878de97 binding host dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 10:12:30.322109 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.349915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.421037 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.429363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-62b17dba-1ef3-4c6d-a9ff-1f89ec74b15a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:30.475483 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroup': ['ID=2b6d1377-7673-408e-86ee-8125745d3311,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:30.478751 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:30.576382 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.578323 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-277a8fd7-10a6-40d8-93ac-2ac2c0415e43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.5009809 May 24 10:12:30.582867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34094) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:30.588875 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:30.595973 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:30.601889 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:30.605450 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:30.612633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:30.708213 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.750077 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks [] for port, precommit_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.768259 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.provisioning_blocks [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Transition to ACTIVE for port object d3ce81dc-f3b2-4725-90b6-77a7e878de97 will not be triggered until provisioned by entity L2. {{(pid=109115) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:30.773014 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:30.817954 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.839145 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.847207 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-06e33fd8-c729-409e-8125-655413d3d8f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:30.859886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:30.860496 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.901546 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:30.914203 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:30.939989 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=96f6fd39-cfb1-4c75-aed4-ffaa4ae3d486,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:30.945509 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:31.033013 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.036126 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-f1546032-77f0-40ea-b354-4a112a898780 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.4513719 May 24 10:12:31.040331 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34098) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:31.048082 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:31.058661 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:31.068272 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:31.073482 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:31.080721 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:31.120726 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a46a7f04-a950-43d9-a167-0d598bc17a76 None None] Pushing event updated for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=3']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:31.161149 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-a46a7f04-a950-43d9-a167-0d598bc17a76 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.314s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:31.165843 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.193293 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Attempting to bind port d3ce81dc-f3b2-4725-90b6-77a7e878de97 on host dv-cinder-791239-2-iscsi for vnic_type normal with profile {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:31.193889 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Attempting to bind port d3ce81dc-f3b2-4725-90b6-77a7e878de97 by drivers openvswitch,hyperv on host dv-cinder-791239-2-iscsi at level 0 using segments [{'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:31.194458 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Attempting to bind port d3ce81dc-f3b2-4725-90b6-77a7e878de97 on network 87e2b5b7-b682-411b-ad8f-70bb178250f4 {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:31.204266 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Checking agent: {'id': 'f204a933-36f4-49bf-be82-70a418e70e72', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-791239-2-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 52, 6), 'started_at': datetime.datetime(2021, 5, 24, 9, 52, 36), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-791239-2-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:31.204798 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Checking segment: {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109115) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:31.205416 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.206009 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Bound using segment: {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'} {{(pid=109115) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:31.206691 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.managers [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Bound port: d3ce81dc-f3b2-4725-90b6-77a7e878de97, host: dv-cinder-791239-2-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9726e60b-1b4d-46d9-9951-468c3217b9eb', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 89, 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4'}}] {{(pid=109115) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:31.207279 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.271789 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.310045 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] For port d3ce81dc-f3b2-4725-90b6-77a7e878de97, host dv-cinder-791239-2-iscsi, cleared binding levels {{(pid=109115) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:31.314064 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.plugins.ml2.db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] For port d3ce81dc-f3b2-4725-90b6-77a7e878de97, host dv-cinder-791239-2-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=d3ce81dc-f3b2-4725-90b6-77a7e878de97,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109115) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:31.341901 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.344046 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ac25db90-0532-4994-9f9e-86333197339d tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:31.369528 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=35f2160f-7b14-406c-90bf-12c0a77e5752,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:31.372115 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:31.392688 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron_lib.callbacks.manager [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109115) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.397608 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:31.443966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.445339 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-4a3d3fc1-d9d9-4d92-bfbc-5c768c9fb431 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.4031425 May 24 10:12:31.450267 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34104) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:31.457918 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:31.469005 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:31.477746 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:31.481868 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:12:31.482181 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:12:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:31.483223 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:31.494845 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:31.559435 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a46a7f04-a950-43d9-a167-0d598bc17a76 None None] Pushing event updated for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=4']} {{(pid=109115) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:31.578083 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG oslo_concurrency.lockutils [None req-a46a7f04-a950-43d9-a167-0d598bc17a76 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.180s {{(pid=109115) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:31.586385 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.696181 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.732417 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:12:31.827551 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:31.847358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:31.890428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=bf53ea6c-089b-4e72-aff1-e575eaef9cf5,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:31.895179 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:32.009487 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.011062 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a736816f-219e-4acf-a7e5-c892bf2904aa tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5594571 May 24 10:12:32.014618 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34108) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:32.022439 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:32.030152 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:32.035134 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:32.038986 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:32.046954 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:32.102261 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:27Z', 'revision_number': 4, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'd3ce81dc-f3b2-4725-90b6-77a7e878de97', 'name': '', 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'mac_address': 'fa:16:3e:c4:5f:93', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_interface', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:26Z', 'updated_at': '2021-05-24T10:12:31Z', 'revision_number': 4, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'subnets': [{'id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:12:32.107155 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a6878d3-40d5-45c1-aaa4-1a033038b15e None None] The sync data for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi is: [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:12:27Z', 'revision_number': 4, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'd3ce81dc-f3b2-4725-90b6-77a7e878de97', 'name': '', 'network_id': '87e2b5b7-b682-411b-ad8f-70bb178250f4', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'mac_address': 'fa:16:3e:c4:5f:93', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_interface', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:26Z', 'updated_at': '2021-05-24T10:12:31Z', 'revision_number': 4, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'subnets': [{'id': '8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:12:32.124823 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.229704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.301693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.303877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:32.333042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d4452b9e-ecb7-4e75-a005-87fc1e8ab98a,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:32.338572 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:32.421537 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.423377 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-05884f89-e022-49aa-a0e9-edad6459b0ed tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.4072447 May 24 10:12:32.427268 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34112) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:32.435220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:32.443784 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:32.450265 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:32.454363 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:32.464925 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:32.541367 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.securitygroups_db [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Project 2cc9bd0e4be64b0e9add9a9f9e8bbe52 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:12:32.551684 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.651652 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.763937 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.767307 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3003daa5-2b94-4fb9-8e52-416046c4273f tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:32.795563 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d3ce81dc-f3b2-4725-90b6-77a7e878de97 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:32.834085 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=4b4376e0-cf3b-40c3-b34c-3862098726b3,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:32.844031 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:32.924283 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:32.926999 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.4975522 May 24 10:12:32.931367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34118) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:12:32.939966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'security_group_rule': {'security_group_id': '2b6d1377-7673-408e-86ee-8125745d3311', 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:32.949305 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:32.955733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:security_group_rule and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:12:32.961436 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:12:32.970009 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:33.009656 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:33.018252 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:33.018252 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.050027 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.securitygroups_db [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Project 2cc9bd0e4be64b0e9add9a9f9e8bbe52 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 10:12:33.059983 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.199167 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.212767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.264246 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3754530'] for security_group_rule, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.268187 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] For port d3ce81dc-f3b2-4725-90b6-77a7e878de97, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=d3ce81dc-f3b2-4725-90b6-77a7e878de97,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:12:33.268522 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a2a03dbf-9a3b-4dea-b397-3a54300f3ca9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:33.299318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=81afdbbe-9559-4af9-87ae-aea08938ab61,revision_number=0']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:33.305518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:33.341150 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b4106f45-e944-4268-b2c2-5578902cefcf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.370311 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.374756 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-6db2aa84-4f2a-433d-971c-a45437da0e1d None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:33.427401 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group_rule, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:33.429469 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0be4665e-d1e5-4044-9e3f-1c670b7870e2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.4976149 May 24 10:12:33.460996 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Pushing event updated for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:33.464519 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:34.759599 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d3ce81dc-f3b2-4725-90b6-77a7e878de97 up at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:12:34.882946 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Provisioning complete for port d3ce81dc-f3b2-4725-90b6-77a7e878de97 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:34.883531 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:35.813599 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-be338859-3ece-437d-a962-6cf3847dcc56 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:35.946703 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:37.249362 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:38.849615 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:50.231961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:51.789493 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-f0d4397d-a0c2-441d-a84b-f786de00d900 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-525824861&fields=id&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 267 time: 0.0464323 May 24 10:12:51.967692 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-5a8c0af4-37a8-4263-8e73-43ef6d49194a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:51.969153 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-5a8c0af4-37a8-4263-8e73-43ef6d49194a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.1715910 May 24 10:12:51.985598 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-00dae943-9caf-4cf9-b7d7-db9419c8f1e1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/quotas/2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 375 time: 0.0100696 May 24 10:12:52.083555 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-a270a2e9-ba51-4fac-bdbc-e9fdfeb5610d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0917320 May 24 10:12:52.101209 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-90045820-0b71-4d6d-b377-6842da2c0570 service neutron] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109113) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:12:52.102255 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-90045820-0b71-4d6d-b377-6842da2c0570 service neutron] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0062253 May 24 10:12:52.251122 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-9fcff2b7-8c25-479c-92b4-f5989a658a98 service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1420631 May 24 10:12:52.398690 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-d6da7364-db9c-4ade-a592-bacf758bd5a7 service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1404071 May 24 10:12:53.372559 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4618dd19-add8-44d1-87ea-ce9184cdeb06 req-0fcba64c-5468-442d-b521-076ab193e94a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:12:53.488939 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4618dd19-add8-44d1-87ea-ce9184cdeb06 req-0fcba64c-5468-442d-b521-076ab193e94a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:12:53.490267 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4618dd19-add8-44d1-87ea-ce9184cdeb06 req-0fcba64c-5468-442d-b521-076ab193e94a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.1655970 May 24 10:12:54.627390 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c890b570-34c6-4e9c-841f-de08afa08db9 req-a9a6565c-0879-4ff7-8513-fce43be685cb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.0567069 May 24 10:12:55.846990 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-56052edf-e0a7-4a53-8c21-90dafa2ef749 req-9c5f43be-4b06-4537-86fb-f7ac8961bdbd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.0435905 May 24 10:12:56.853150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-a6165fe2-3075-4a40-b5b0-7fa5b436e5d5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:56.854572 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-a6165fe2-3075-4a40-b5b0-7fa5b436e5d5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.1911876 May 24 10:12:56.911895 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-ed497d06-726a-4cd0-8b6a-4cb48966e1c6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 379 time: 0.0454466 May 24 10:12:56.921737 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'port': {'device_id': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'admin_state_up': True, 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'security_groups': ['2b6d1377-7673-408e-86ee-8125745d3311']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:57.061785 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-71159f92-5485-4789-82e9-a60b172c9b92 req-4b3e6186-81b8-41c1-86ac-b5b211bf9e36 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.0987585 May 24 10:12:57.130268 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:12:57.145905 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:12:57.259861 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Made reservation on behalf of 2cc9bd0e4be64b0e9add9a9f9e8bbe52 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:12:57.264478 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:57.443651 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.20.0.3 (3670d884-4bc4-4d4e-8dbb-1165ed224ea7/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51/deb1beef-4279-4eb7-8f77-beec25a50790) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:12:57.790806 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:57.807691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Transition to ACTIVE for port object deb1beef-4279-4eb7-8f77-beec25a50790 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:57.969635 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:57.975771 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-725230f3-477b-4655-b45b-b175c3761e10 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:58.042765 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:12:58.045998 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:12:58.049907 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:12:58.089872 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.113688 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.114527 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:12:58.119734 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:58.145601 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:58.155948 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:58.328123 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c1147fe-baf1-4edd-ad4a-4cddf9622e30 req-9dcc7493-6efd-433f-bcca-02d4c3b4a153 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:58.329809 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-171917d5-0d94-4964-82ff-01be944bf398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4121616 May 24 10:12:58.330955 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7c1147fe-baf1-4edd-ad4a-4cddf9622e30 req-9dcc7493-6efd-433f-bcca-02d4c3b4a153 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 931 time: 0.0983286 May 24 10:12:58.345033 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Request body: {'port': {'device_id': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:12:58.400831 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-7c1147fe-baf1-4edd-ad4a-4cddf9622e30 req-1ddf4da3-344b-485d-bc0d-90e4be2317eb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:security_group {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:12:58.499853 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-55990e37-0c2f-4d24-82d0-9ac72382038b None None] Provisioning for port deb1beef-4279-4eb7-8f77-beec25a50790 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:12:58.506284 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-55990e37-0c2f-4d24-82d0-9ac72382038b None None] Provisioning complete for port deb1beef-4279-4eb7-8f77-beec25a50790 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:12:58.508965 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-55990e37-0c2f-4d24-82d0-9ac72382038b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:58.536735 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-7c1147fe-baf1-4edd-ad4a-4cddf9622e30 req-1ddf4da3-344b-485d-bc0d-90e4be2317eb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:security_group {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:12:58.538688 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7c1147fe-baf1-4edd-ad4a-4cddf9622e30 req-1ddf4da3-344b-485d-bc0d-90e4be2317eb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2004974 May 24 10:12:58.561428 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:58.590989 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-55990e37-0c2f-4d24-82d0-9ac72382038b None None] Port deb1beef-4279-4eb7-8f77-beec25a50790 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:12:59.022849 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id deb1beef-4279-4eb7-8f77-beec25a50790 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:12:59.031640 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:59.048345 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Transition to ACTIVE for port object deb1beef-4279-4eb7-8f77-beec25a50790 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:12:59.168819 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:59.173250 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-e8084643-4532-4dd1-b717-73fd4731a195 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:59.336868 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Pushing event updated for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:59.344953 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:59.433401 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Attempting to bind port deb1beef-4279-4eb7-8f77-beec25a50790 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:12:59.433733 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Attempting to bind port deb1beef-4279-4eb7-8f77-beec25a50790 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:12:59.434062 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Attempting to bind port deb1beef-4279-4eb7-8f77-beec25a50790 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:59.438945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Port deb1beef-4279-4eb7-8f77-beec25a50790 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:12:59.439254 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Attempting to bind port deb1beef-4279-4eb7-8f77-beec25a50790 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:12:59.444456 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 12, 50), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:12:59.444767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:12:59.445196 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Bound using segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:12:59.445620 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Bound port: deb1beef-4279-4eb7-8f77-beec25a50790, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:12:59.445971 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:59.543234 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:12:59.546610 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=deb1beef-4279-4eb7-8f77-beec25a50790,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:12:59.704568 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:59.715129 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d15b057b-524a-4c3e-bbc3-7034489acbbf tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:12:59.788031 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:59.788720 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:12:59.790143 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] 10.222.0.95 "PUT /v2.0/ports/deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 200 len: 1093 time: 1.4519620 May 24 10:12:59.790720 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:12:59.821390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Pushing event updated for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:12:59.825835 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:12:59.845464 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f7e5b1a-c023-4451-9bcf-72276b82896a req-4f242f64-a465-4b61-a7cb-749a7be2e3a7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:59.846264 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3f7e5b1a-c023-4451-9bcf-72276b82896a req-4f242f64-a465-4b61-a7cb-749a7be2e3a7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:12:59.850681 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'status': 'completed', 'code': 200} May 24 10:12:59.951325 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-b266e851-7182-4fe9-8407-5d0f5ec61d77 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:12:59.953610 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-b266e851-7182-4fe9-8407-5d0f5ec61d77 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1096 time: 0.0887475 May 24 10:13:00.053515 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-3f7e5b1a-c023-4451-9bcf-72276b82896a req-4f242f64-a465-4b61-a7cb-749a7be2e3a7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:13:00.054711 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3f7e5b1a-c023-4451-9bcf-72276b82896a req-4f242f64-a465-4b61-a7cb-749a7be2e3a7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.3908839 May 24 10:13:00.142151 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-3f7e5b1a-c023-4451-9bcf-72276b82896a req-2442c7ae-ca5f-4c09-b56e-aed368280f39 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0788100 May 24 10:13:00.145747 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-fd7bd62c-8b8c-491a-8fa5-3c6e148da2db service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:00.147215 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-fd7bd62c-8b8c-491a-8fa5-3c6e148da2db service neutron] 10.222.0.95 "GET /v2.0/networks?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&shared=False HTTP/1.1" status: 200 len: 912 time: 0.1865809 May 24 10:13:00.291176 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-9012d08c-91f6-487c-805f-8a09cc8f7dbe service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:00.292632 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b4d90c76-5263-498c-8ea1-8c6bfc5f7cda req-9012d08c-91f6-487c-805f-8a09cc8f7dbe service neutron] 10.222.0.95 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 869 time: 0.1389787 May 24 10:13:00.407814 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-7eb87428-9cba-4ad4-927b-55f7a4595363 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:00.410082 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-7eb87428-9cba-4ad4-927b-55f7a4595363 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1096 time: 0.0779407 May 24 10:13:00.609083 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-d5b9eabe-9aa1-44d2-9aee-386130b768aa service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 200 len: 217 time: 0.1911814 May 24 10:13:00.674654 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-67cc772d-6920-4c89-ae7b-32f35890d0e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:00.676052 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-67cc772d-6920-4c89-ae7b-32f35890d0e5 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0606790 May 24 10:13:00.773507 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4c6e7a55-1061-4bc9-a137-bb28d236dcc3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:00.774991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-4c6e7a55-1061-4bc9-a137-bb28d236dcc3 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0924816 May 24 10:13:00.918933 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-cb78fbc6-623e-437c-b77a-338588ab777c service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1365039 May 24 10:13:01.067845 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9aaea6fb-7d52-43a4-8d19-9cb835ebddda req-bfea4ebc-3adf-4001-8fb8-9f1afea2ce83 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1410649 May 24 10:13:01.357912 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a84f66be-5403-4b6c-a16c-c05d28b07e4b req-3ea5f6fd-e3fd-476f-b9da-e98dfc7f980b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:01.359391 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a84f66be-5403-4b6c-a16c-c05d28b07e4b req-3ea5f6fd-e3fd-476f-b9da-e98dfc7f980b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0951471 May 24 10:13:01.411352 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a84f66be-5403-4b6c-a16c-c05d28b07e4b req-995d1d0d-29a4-4e39-ac54-281b37d6da35 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0453801 May 24 10:13:02.609301 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e3f87d1-70aa-4af1-b331-6393527c2492 req-00b124de-e524-4ac3-af17-1636eedfbf44 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:02.611189 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e3f87d1-70aa-4af1-b331-6393527c2492 req-00b124de-e524-4ac3-af17-1636eedfbf44 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0870800 May 24 10:13:02.658517 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4e3f87d1-70aa-4af1-b331-6393527c2492 req-efd8a21a-15c3-4d3f-ab70-35ef9126be51 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411732 May 24 10:13:03.374007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-32ff6c2e-5502-4ee0-b338-57e82f18d1bc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:03.861452 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3da82c1-9671-4925-8475-0d7dcf6bdf5f req-bb70906d-f2f1-44df-bf46-c8331ba4ef43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:03.863653 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a3da82c1-9671-4925-8475-0d7dcf6bdf5f req-bb70906d-f2f1-44df-bf46-c8331ba4ef43 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0890183 May 24 10:13:03.916394 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a3da82c1-9671-4925-8475-0d7dcf6bdf5f req-9811ca62-0b07-4ba4-94d0-e28305634eb0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0462978 May 24 10:13:04.494229 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:13:05.143122 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-349d9bdc-62cf-4ba8-bc4e-d87aa7afa636 req-02b6990c-5dfe-42dd-b8e8-dd27b690e9b3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:05.145961 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-349d9bdc-62cf-4ba8-bc4e-d87aa7afa636 req-02b6990c-5dfe-42dd-b8e8-dd27b690e9b3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.1125522 May 24 10:13:05.196439 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-349d9bdc-62cf-4ba8-bc4e-d87aa7afa636 req-653327a8-8fa3-4f44-a7e8-49feaac39a24 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0445910 May 24 10:13:05.817000 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ff01e5a0-fa6e-4c20-be21-e1ca4539cb8f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:05.900024 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:06.406804 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b70fd4c-44e5-459a-ae1f-2ee1f975d2e1 req-4488077e-40eb-4644-94a8-4343728caebf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:06.408376 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8b70fd4c-44e5-459a-ae1f-2ee1f975d2e1 req-4488077e-40eb-4644-94a8-4343728caebf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0879073 May 24 10:13:06.563316 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-8b70fd4c-44e5-459a-ae1f-2ee1f975d2e1 req-03733bf6-6316-42eb-9b91-de965c4414f3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1485505 May 24 10:13:07.270877 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:07.774252 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a597da2-fedd-4296-8a2d-9e02458e44e8 req-bab367ad-0ab3-4870-b7ff-b1d335e540d5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:07.776011 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9a597da2-fedd-4296-8a2d-9e02458e44e8 req-bab367ad-0ab3-4870-b7ff-b1d335e540d5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0877659 May 24 10:13:07.833698 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9a597da2-fedd-4296-8a2d-9e02458e44e8 req-e52a5d60-6dab-4e94-91da-d3806e2fba16 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0476532 May 24 10:13:08.802840 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:09.043094 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32b2b8e4-f3a1-4bc3-a4c9-24184df163c8 req-bb0ab1e8-5dd8-4f11-86ce-2b8420520d12 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:09.044692 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-32b2b8e4-f3a1-4bc3-a4c9-24184df163c8 req-bb0ab1e8-5dd8-4f11-86ce-2b8420520d12 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 943 time: 0.0959640 May 24 10:13:09.094795 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-32b2b8e4-f3a1-4bc3-a4c9-24184df163c8 req-61138dc2-98ca-45b7-a29c-cd5efbff3bb3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0427492 May 24 10:13:09.605524 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device deb1beef-4279-4eb7-8f77-beec25a50790 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:13:09.816738 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=deb1beef-4279-4eb7-8f77-beec25a50790,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:13:09.830602 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'port_id': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'mac_address': 'fa:16:3e:7f:23:16', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:13:09.896365 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:09.914846 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id deb1beef-4279-4eb7-8f77-beec25a50790 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:13:09.932826 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=deb1beef-4279-4eb7-8f77-beec25a50790,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:13:10.014039 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:10.016461 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:13:10.094457 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:13:10.104009 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.109767 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:13:10.113030 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:13:10.277611 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['deb1beef-4279-4eb7-8f77-beec25a50790'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:13:10.315084 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d45b58b7-ab53-4179-99b9-d19d4717a4fe req-8a40e200-f5f0-4ff3-aeea-c1e7d3539545 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:10.317773 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d45b58b7-ab53-4179-99b9-d19d4717a4fe req-8a40e200-f5f0-4ff3-aeea-c1e7d3539545 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 944 time: 0.1046739 May 24 10:13:10.392457 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d45b58b7-ab53-4179-99b9-d19d4717a4fe req-59a641fa-68dc-4a34-b987-3592c339ca94 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0678585 May 24 10:13:11.611314 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbcd1444-2ee3-42d5-aeb5-5dd30742ab07 req-1d7f619b-70f5-4caf-9e3c-540918a62b52 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:11.614187 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cbcd1444-2ee3-42d5-aeb5-5dd30742ab07 req-1d7f619b-70f5-4caf-9e3c-540918a62b52 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 944 time: 0.1034465 May 24 10:13:11.614421 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device deb1beef-4279-4eb7-8f77-beec25a50790 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:13:11.673240 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-cbcd1444-2ee3-42d5-aeb5-5dd30742ab07 req-667f5daf-87d1-4903-8374-9c80afb645df tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0484011 May 24 10:13:11.745474 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port deb1beef-4279-4eb7-8f77-beec25a50790 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:13:11.751218 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port deb1beef-4279-4eb7-8f77-beec25a50790 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:13:11.751614 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:11.897791 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:11.928811 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:13:11.943490 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=deb1beef-4279-4eb7-8f77-beec25a50790,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:13:12.042437 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:12.044721 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:13:12.110676 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'code': 200} May 24 10:13:12.175908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:13:12.179520 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:13:12.560135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a6162eb-06f8-43dc-a2ed-dad2afeef798 req-50efaee2-46cc-409d-8c0b-2a32aaadc8d2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:12.561693 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9a6162eb-06f8-43dc-a2ed-dad2afeef798 req-50efaee2-46cc-409d-8c0b-2a32aaadc8d2 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1098 time: 0.0873442 May 24 10:13:12.607419 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9a6162eb-06f8-43dc-a2ed-dad2afeef798 req-25f7046e-3f38-45d7-a984-acbacb962db6 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386446 May 24 10:13:12.890001 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d9f1b90-90a5-457a-a950-524e9c26330c req-295afd82-a8f8-442f-996f-422498d00e40 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:12.891492 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3d9f1b90-90a5-457a-a950-524e9c26330c req-295afd82-a8f8-442f-996f-422498d00e40 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0876973 May 24 10:13:12.948324 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-3d9f1b90-90a5-457a-a950-524e9c26330c req-0becc77b-6e5d-4ef9-8e83-814b1a7d8f90 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0496712 May 24 10:13:14.164267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b2a6f263-4440-4805-966c-1aa3f46849b5 req-7cb29e80-fb39-41f0-83ca-a3bfcc411d44 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:14.166128 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2a6f263-4440-4805-966c-1aa3f46849b5 req-7cb29e80-fb39-41f0-83ca-a3bfcc411d44 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0998030 May 24 10:13:14.219690 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-b2a6f263-4440-4805-966c-1aa3f46849b5 req-a0d4bd94-bba6-4af8-9845-2a5719281e6e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475724 May 24 10:13:14.429958 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc8b88f1-d846-40a8-b951-3b514ee8c6c0 req-0c1f54a5-17a1-4796-a749-c1a99e0e7be5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:14.432526 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bc8b88f1-d846-40a8-b951-3b514ee8c6c0 req-0c1f54a5-17a1-4796-a749-c1a99e0e7be5 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0937753 May 24 10:13:14.485183 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-bc8b88f1-d846-40a8-b951-3b514ee8c6c0 req-01edb033-c149-4e9e-93f8-de954e78c54d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0462930 May 24 10:13:14.511406 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34300) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:13:14.760425 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-246c4623-29c4-4bd3-9431-ad768aab4d7d tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:14.761802 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-246c4623-29c4-4bd3-9431-ad768aab4d7d tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] Synchronizing usage tracker for tenant:0850b91daf724bcd93114c2b6acaf926 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:13:14.876742 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-246c4623-29c4-4bd3-9431-ad768aab4d7d tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] Unset dirty status for tenant:0850b91daf724bcd93114c2b6acaf926 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:13:14.878523 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-246c4623-29c4-4bd3-9431-ad768aab4d7d tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1093 time: 0.3654513 May 24 10:13:14.882304 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34306) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:13:14.888125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'port_id': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address': '172.20.0.3'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:13:14.895168 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:13:14.899537 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:floatingip and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:13:14.902792 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=109113) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:13:14.910043 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:13:14.996866 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:15.173286 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:15.348190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.27.0.205 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/a5af3e6d-7429-4272-bdf8-b962bdea1b93) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:13:15.568667 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:15.741150 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:15.746375 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-28d26eb0-3bc0-469b-839e-9fcb7784b190 tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:13:15.821482 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:13:15.824851 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:13:15.828119 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:13:15.871915 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=a5af3e6d-7429-4272-bdf8-b962bdea1b93,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:13:15.878528 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:13:16.495221 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:16.672026 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:16.904394 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:17.042408 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:17.046642 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5b4bc992-a982-4047-9a3c-703a4d60c6af service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:13:17.160413 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=a5af3e6d-7429-4272-bdf8-b962bdea1b93,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:13:17.165085 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:13:17.175245 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:17.377473 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 6ed2d069-1420-4918-b860-85d659542cc3 associated. External IP: 172.27.0.205, port: deb1beef-4279-4eb7-8f77-beec25a50790. May 24 10:13:17.761360 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:13:18.103594 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:13:18.116886 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:13:18.130779 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:13:18.131566 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:13:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:13:18.219351 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:18.302723 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-0d8fda89-b3c6-4ed0-a8aa-2e1985a486d2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.4192343 May 24 10:13:18.303406 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:13:18.341168 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:13:18.356352 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'status': 'completed', 'code': 200} May 24 10:13:18.460125 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22091b59-2cda-4058-a7dd-c251250399fa req-1dea7189-30fc-49a9-b4e2-e423cea0df1e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:18.462130 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-1dea7189-30fc-49a9-b4e2-e423cea0df1e service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1098 time: 0.0818770 May 24 10:13:18.695987 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22091b59-2cda-4058-a7dd-c251250399fa req-bd1cf5fc-1222-4ceb-824a-7b3222643595 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:18.698495 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-bd1cf5fc-1222-4ceb-824a-7b3222643595 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.2293460 May 24 10:13:18.889501 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '6ed2d069-1420-4918-b860-85d659542cc3', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.205', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:7f:23:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:13:16Z', 'updated_at': '2021-05-24T10:13:16Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:13:18.894881 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:13:18.901897 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-b4a2d44a-b9e5-490e-b9d7-a3ee078363b7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 200 len: 999 time: 0.1965060 May 24 10:13:18.905951 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:13:18.906768 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:13:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:13:19.017991 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22091b59-2cda-4058-a7dd-c251250399fa req-1eedd0ab-8f0a-4757-ae89-12a9b26c31fe service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:19.020935 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-1eedd0ab-8f0a-4757-ae89-12a9b26c31fe service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.1122360 May 24 10:13:19.108387 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:13:19.141923 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22091b59-2cda-4058-a7dd-c251250399fa req-4e167d3d-06df-4bbe-b377-adcdb527bf02 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:19.144689 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-4e167d3d-06df-4bbe-b377-adcdb527bf02 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1132216 May 24 10:13:19.306867 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-848d4dba-0e65-4cad-8a07-2ab4c267daef service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1547825 May 24 10:13:19.461458 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-22091b59-2cda-4058-a7dd-c251250399fa req-0eb53c03-3dc3-4685-ae43-f310c7bb55e6 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1436634 May 24 10:13:19.633319 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '6ed2d069-1420-4918-b860-85d659542cc3', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.205', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:7f:23:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:13:16Z', 'updated_at': '2021-05-24T10:13:16Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:13:19.638866 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bf51b976-2511-4f3c-9cde-b4cbfdde5561 None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '6ed2d069-1420-4918-b860-85d659542cc3', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.205', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 681, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:7f:23:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:13:16Z', 'updated_at': '2021-05-24T10:13:16Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:13:19.848982 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19fb5843-dfcd-4048-b892-abde381b1a02 None None] New status for floating IP 6ed2d069-1420-4918-b860-85d659542cc3: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:13:20.293890 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:33.322378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-91ca6128-00ae-43ff-9767-eb74c8be1e47 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:35.841901 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-62e9af04-f95b-42d3-829f-0d7f409f578e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:35.932908 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:37.316318 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:38.812937 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:41.494405 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:13:50.280937 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:13:53.447529 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-918ddaeb-e72a-4425-9e07-baa5d8f8bb76 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0758462 May 24 10:13:53.606334 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-86ac8e4a-9f08-4bbf-b1ca-ffcbd0f31023 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:53.607914 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-86ac8e4a-9f08-4bbf-b1ca-ffcbd0f31023 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1098 time: 0.0818360 May 24 10:13:53.769028 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-2427ab90-7942-4210-bd5c-e0cb93c468bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:53.770530 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-2427ab90-7942-4210-bd5c-e0cb93c468bf service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1404378 May 24 10:13:53.922465 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-6a19f61a-43da-4288-8259-b81c1fc7cb3c service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 200 len: 1001 time: 0.1429515 May 24 10:13:53.992336 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-acde2b0f-b4eb-4a08-8855-7035c34065c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:53.993946 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-acde2b0f-b4eb-4a08-8855-7035c34065c9 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0637426 May 24 10:13:54.075644 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-798e1cd6-426e-4df3-bf1d-a3d60217dd24 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:13:54.077190 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-798e1cd6-426e-4df3-bf1d-a3d60217dd24 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0760815 May 24 10:13:54.223156 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-1f588d63-d001-4859-bc4d-2fe1cee503b1 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1385615 May 24 10:13:54.502816 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a35f6ee9-10c2-4049-88e5-fc805db5ab98 req-11b04acf-8ecc-49f1-8ed6-6e116f23a306 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2727814 May 24 10:14:03.303822 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3662d18f-b0ab-4559-9f4f-1b878a05d4e5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:05.846925 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-12a2d88d-3918-4b71-a96b-d132f5f47a50 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:05.943885 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:07.234634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:08.811789 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:18.494450 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:14:20.262707 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:33.336772 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-d8293adb-3ed8-41bb-8520-820d5de9ef1d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:35.862970 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e33a3ca6-4956-4147-a623-da32171a2347 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:35.908483 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:37.256948 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:38.845969 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:50.290481 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:14:55.471192 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-e47c2e32-37b4-45a7-8698-c093d0ff5001 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0866103 May 24 10:14:55.494895 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:14:55.636947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-f82dcf70-3516-4dd8-976b-ba587ba18c8f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:14:55.639080 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-f82dcf70-3516-4dd8-976b-ba587ba18c8f service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 1098 time: 0.0938182 May 24 10:14:55.810305 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-3d44ce54-24e7-4ea4-9e55-85e856353ad6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:14:55.812500 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-3d44ce54-24e7-4ea4-9e55-85e856353ad6 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1469579 May 24 10:14:55.924942 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-c22b15e5-b3e1-46fd-8724-e8250d7f0a33 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 200 len: 1001 time: 0.1062210 May 24 10:14:55.998661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-8c845579-4031-4a74-8635-438f8babc159 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:14:56.001198 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-8c845579-4031-4a74-8635-438f8babc159 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0697782 May 24 10:14:56.089441 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-a58aff33-d1be-4788-baea-c07e0f55cd78 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:14:56.090987 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-a58aff33-d1be-4788-baea-c07e0f55cd78 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0836294 May 24 10:14:56.244407 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-e49da25d-a1d3-4a8f-942f-d9c242fe3fb8 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1444051 May 24 10:14:56.413546 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-457f7ba9-703b-455f-b7a3-d7ca6c29c4b3 req-e64b2151-325b-4645-9749-dfac344b4cd5 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1619837 May 24 10:15:03.336474 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-f10d6c41-5a01-49d4-ac0f-740b109b2f16 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:05.879037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8dd7f816-fca7-4ab0-bade-9119916dd7b6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:05.959100 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:07.278608 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:08.821543 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:14.102775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-349a7f8b-1e6f-4aa3-9cb3-7c4f28a5735d req-db19196a-36bd-405d-8718-7d2424347fa2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:14.104480 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-349a7f8b-1e6f-4aa3-9cb3-7c4f28a5735d req-db19196a-36bd-405d-8718-7d2424347fa2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0963936 May 24 10:15:14.161529 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-349a7f8b-1e6f-4aa3-9cb3-7c4f28a5735d req-dee58b08-9932-4bc5-98b5-b9efd90e2bcd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0465291 May 24 10:15:15.404510 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0690d35e-d7ee-4208-80f5-55201acdd571 req-bf672cd5-e0ba-4556-8e33-96d4c3daaade tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:15.407009 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0690d35e-d7ee-4208-80f5-55201acdd571 req-bf672cd5-e0ba-4556-8e33-96d4c3daaade tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0954659 May 24 10:15:15.463113 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-0690d35e-d7ee-4208-80f5-55201acdd571 req-3ac30834-82aa-4dd3-b9a3-b9a177159d6b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461428 May 24 10:15:15.635830 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-773891dd-f7de-4d7c-8b90-6c0bca9f3beb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:15.637495 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-773891dd-f7de-4d7c-8b90-6c0bca9f3beb tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 945 time: 0.0875471 May 24 10:15:15.806785 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port deb1beef-4279-4eb7-8f77-beec25a50790 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:15:15.807126 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:15.948705 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] For port deb1beef-4279-4eb7-8f77-beec25a50790, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=deb1beef-4279-4eb7-8f77-beec25a50790,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:15:15.949160 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:16.379949 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:16.380318 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:16.380659 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 6ed2d069-1420-4918-b860-85d659542cc3 disassociated. External IP: 172.27.0.205, port: deb1beef-4279-4eb7-8f77-beec25a50790. May 24 10:15:16.381001 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for deb1beef-4279-4eb7-8f77-beec25a50790 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:15:16.582062 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:16.591629 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-ef858970-805d-4e10-9c17-1e74cca10319 tempest-AttachVolumeShelveTestJSON-1229989023 tempest-AttachVolumeShelveTestJSON-1229989023-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:16.685646 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=deb1beef-4279-4eb7-8f77-beec25a50790,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:16.698360 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e713c97d-fbe0-4e9c-a37e-e969e63bcdf4 req-54e28c69-746d-400f-bebc-260203991e12 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.0700350 May 24 10:15:16.705939 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.706328 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.709655 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.711080 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:16.714095 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:16.918876 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:15:17.145253 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:15:17.160210 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:15:17.181812 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:15:17.182746 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:15:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:17.271796 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] No DVR routers for this DVR port deb1beef-4279-4eb7-8f77-beec25a50790 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:15:17.274746 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:17.278003 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'name': 'network-vif-deleted', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:15:17.361606 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'add96aba-6f75-4d91-b87a-8fe1918c02ab', 'name': 'network-vif-deleted', 'tag': 'deb1beef-4279-4eb7-8f77-beec25a50790', 'status': 'completed', 'code': 200} May 24 10:15:17.365922 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-e51eeca4-5c35-4ed9-8b66-86fb1a2258c6 req-d59faeda-2ea3-4831-af81-397f6a06debe tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "DELETE /v2.0/ports/deb1beef-4279-4eb7-8f77-beec25a50790 HTTP/1.1" status: 204 len: 173 time: 1.7221923 May 24 10:15:17.436913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:15:17.847603 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:17.854139 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:15:17.875701 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:15:17.877595 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:15:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:17.960406 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-f7b1f6fd-037b-4d87-9ca7-b856a073546f req-420b072d-d3cb-42e9-88f3-d6c4f2cdbb15 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:15:18.072691 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-f7b1f6fd-037b-4d87-9ca7-b856a073546f req-420b072d-d3cb-42e9-88f3-d6c4f2cdbb15 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:15:18.073956 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f7b1f6fd-037b-4d87-9ca7-b856a073546f req-420b072d-d3cb-42e9-88f3-d6c4f2cdbb15 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=add96aba-6f75-4d91-b87a-8fe1918c02ab HTTP/1.1" status: 200 len: 210 time: 0.1966662 May 24 10:15:18.219620 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:15:18.580555 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:18.585466 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2cc17bd2-71c2-4fff-8bd5-6707d7971457 None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:15:18.855056 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d573c240-50be-4040-b829-f70db59a8b86 None None] New status for floating IP 6ed2d069-1420-4918-b860-85d659542cc3: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:15:19.528786 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-aef6154f-e941-49c6-8bf0-2d135509c0e7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-525824861&fields=id&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 267 time: 0.0740690 May 24 10:15:19.730051 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-f911bfaa-3a9a-4651-8b64-2cb9cfbb4d06 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:19.731675 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-f911bfaa-3a9a-4651-8b64-2cb9cfbb4d06 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.1946859 May 24 10:15:19.751764 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-ad055fd4-0eaf-483a-9739-6b23edae5392 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/quotas/2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 375 time: 0.0115526 May 24 10:15:19.853248 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-9d4d10f9-aa4a-4f08-8c7e-be9ed2f340fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0948999 May 24 10:15:20.076008 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-a11d6dcf-8a74-42b1-8696-a0b26a565ee5 service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2088461 May 24 10:15:20.231574 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-003dfa18-bae5-4d74-b30a-be43422bb875 service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1483750 May 24 10:15:20.241817 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device deb1beef-4279-4eb7-8f77-beec25a50790 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:15:20.308411 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:20.312662 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: deb1beef-4279-4eb7-8f77-beec25a50790 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:15:20.312662 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device deb1beef-4279-4eb7-8f77-beec25a50790 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:15:21.012204 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e2430ed0-011e-4b26-8946-42a21221ee5f req-8c3195c7-5afb-4c28-a376-beee33b3ebf1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 210 time: 0.0534825 May 24 10:15:22.225708 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-944bb877-84fe-4048-be9b-c28c316de7c9 req-5c85cc02-48e3-4ed1-b318-1c1d2d3b1091 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 210 time: 0.0485892 May 24 10:15:23.434478 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b458347b-4e20-4603-a039-d2089654b168 req-5b47db8e-8ae3-47e5-9fa4-e661557905ef tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 210 time: 0.0723286 May 24 10:15:23.833391 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-18331f8d-0f39-418a-89d5-3fbe5c22cb91 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:23.834860 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-18331f8d-0f39-418a-89d5-3fbe5c22cb91 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.3355610 May 24 10:15:23.895163 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-1aa85406-6aee-4614-873c-f58268fe420a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 379 time: 0.0507324 May 24 10:15:23.904068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'port': {'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'admin_state_up': True, 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'security_groups': ['2b6d1377-7673-408e-86ee-8125745d3311']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:15:24.060653 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:15:24.073526 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:15:24.159819 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Made reservation on behalf of 2cc9bd0e4be64b0e9add9a9f9e8bbe52 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:15:24.164295 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:24.364453 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.20.0.12 (3670d884-4bc4-4d4e-8dbb-1165ed224ea7/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51/2faafcb9-9d9d-4bf2-997a-509be060bb30) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:15:24.627437 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-a28572e0-684b-4f05-b007-ef61bda1b993 req-4201ce2c-b30a-45d8-83cb-5dfe9403833b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 210 time: 0.0653942 May 24 10:15:24.641177 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:24.664061 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Transition to ACTIVE for port object 2faafcb9-9d9d-4bf2-997a-509be060bb30 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:15:24.782921 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:24.789374 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-4b27b268-e662-4055-aedb-c060c06d26f6 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:24.884280 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:15:24.892055 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:15:24.893027 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:15:24.927225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:24.934402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.145s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:24.973375 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:24.974133 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:25.159305 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-5d257b44-9e13-4317-894f-64892d1b0b9e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.2590744 May 24 10:15:25.171999 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Request body: {'port': {'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:15:25.261228 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6d8420b8-4d80-4e00-89cd-e71896c5b38c None None] Provisioning for port 2faafcb9-9d9d-4bf2-997a-509be060bb30 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:15:25.268014 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-6d8420b8-4d80-4e00-89cd-e71896c5b38c None None] Provisioning complete for port 2faafcb9-9d9d-4bf2-997a-509be060bb30 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:15:25.268546 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-6d8420b8-4d80-4e00-89cd-e71896c5b38c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:25.300747 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:25.332068 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-6d8420b8-4d80-4e00-89cd-e71896c5b38c None None] Port 2faafcb9-9d9d-4bf2-997a-509be060bb30 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:15:25.763104 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 2faafcb9-9d9d-4bf2-997a-509be060bb30 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:15:25.776955 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:25.807560 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Transition to ACTIVE for port object 2faafcb9-9d9d-4bf2-997a-509be060bb30 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:15:25.877930 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:25.883595 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-172530eb-f396-4dbb-ba2f-3c1ce8b1577f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:25.942623 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7217334-6467-449f-afcc-2a3373869af1 req-7fa324d6-b92f-449c-b4be-1caed0cd90b3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:25.945294 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e7217334-6467-449f-afcc-2a3373869af1 req-7fa324d6-b92f-449c-b4be-1caed0cd90b3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 932 time: 0.1235859 May 24 10:15:26.034885 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-e7217334-6467-449f-afcc-2a3373869af1 req-b18db15c-12c2-4c0e-a18a-cb230fbbd637 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0780799 May 24 10:15:26.072871 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Pushing event updated for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:26.081355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.198s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:26.207905 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Attempting to bind port 2faafcb9-9d9d-4bf2-997a-509be060bb30 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:15:26.208254 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Attempting to bind port 2faafcb9-9d9d-4bf2-997a-509be060bb30 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:15:26.208559 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Attempting to bind port 2faafcb9-9d9d-4bf2-997a-509be060bb30 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:15:26.214355 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Port 2faafcb9-9d9d-4bf2-997a-509be060bb30 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:15:26.214700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Attempting to bind port 2faafcb9-9d9d-4bf2-997a-509be060bb30 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:15:26.221933 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 15, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:15:26.222292 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:15:26.222722 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Bound using segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:15:26.223190 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Bound port: 2faafcb9-9d9d-4bf2-997a-509be060bb30, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:15:26.223669 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:26.327393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:15:26.331185 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:15:26.432766 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:26.437191 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b87531c7-0e5f-4547-a4bb-ba777aa4f180 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:26.507600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:26.508539 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:26.510872 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] 10.222.0.95 "PUT /v2.0/ports/2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 1094 time: 1.3437505 May 24 10:15:26.511700 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:15:26.545470 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Pushing event updated for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:26.549333 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:26.611744 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'status': 'completed', 'code': 200} May 24 10:15:26.822163 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-08bbdfa9-458b-4c3a-ad54-51e01cbec3c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:26.824084 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-08bbdfa9-458b-4c3a-ad54-51e01cbec3c6 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1097 time: 0.2085993 May 24 10:15:26.903203 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-2e1a04e9-2a6f-4e81-ba82-027fa2d30899 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 217 time: 0.0708680 May 24 10:15:26.983616 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-57dbf998-d17e-4bc0-88e9-67c268dfd904 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:26.985244 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-57dbf998-d17e-4bc0-88e9-67c268dfd904 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0751295 May 24 10:15:27.085283 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68d6c1af-3672-4785-a200-707987fff509 req-e9d2787e-d5d9-4360-b3da-4e55448b1d6e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:27.088349 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-e9d2787e-d5d9-4360-b3da-4e55448b1d6e service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0935581 May 24 10:15:27.316387 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a70c075f-e643-45df-bad8-83cfb66cd6c7 req-524b048d-f966-4ea8-93da-4553fa656d2b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:27.318145 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a70c075f-e643-45df-bad8-83cfb66cd6c7 req-524b048d-f966-4ea8-93da-4553fa656d2b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.1075611 May 24 10:15:27.341284 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-b3f7f0cf-631e-4000-937f-dac0e5e7e349 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2453821 May 24 10:15:27.381461 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-a70c075f-e643-45df-bad8-83cfb66cd6c7 req-743522df-490a-4aef-a120-a7390cdf3241 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0567772 May 24 10:15:27.575973 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-68d6c1af-3672-4785-a200-707987fff509 req-cd271c40-9ee0-49e2-b431-89623d6de976 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2279804 May 24 10:15:27.834978 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-0e9fa74b-f0e7-48ee-8aff-2b0239f6781d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:27.837554 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-0e9fa74b-f0e7-48ee-8aff-2b0239f6781d service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1097 time: 0.0829248 May 24 10:15:27.981308 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-3d5045f3-a3f0-4f22-ae96-067910785ba7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:27.983406 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-3d5045f3-a3f0-4f22-ae96-067910785ba7 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1385543 May 24 10:15:28.056043 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-c9f2f30e-7709-487c-b8fa-e5c72a071ad0 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 217 time: 0.0652134 May 24 10:15:28.122964 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-e91d0e93-2de1-4327-80e8-6318d12009ac service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:28.124363 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-e91d0e93-2de1-4327-80e8-6318d12009ac service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0620880 May 24 10:15:28.207516 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-de3ad7b1-3387-4c96-8ae4-fa78fe942f65 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:28.209099 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-de3ad7b1-3387-4c96-8ae4-fa78fe942f65 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0783949 May 24 10:15:28.341566 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-75c1cade-9164-4eb0-9480-c484ec1603e6 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1256723 May 24 10:15:28.483834 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-700b25be-2bf5-4dc9-956a-924bd0cf401a req-0b76b10b-1e1a-462f-8238-d94ff7d073e6 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1356149 May 24 10:15:28.583424 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26b3eee0-ce98-444a-9f7a-1abd440b5561 req-0fc91425-2e06-4bc6-a6e8-c84756983ac7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:28.584158 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-26b3eee0-ce98-444a-9f7a-1abd440b5561 req-0fc91425-2e06-4bc6-a6e8-c84756983ac7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:15:28.661457 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-26b3eee0-ce98-444a-9f7a-1abd440b5561 req-0fc91425-2e06-4bc6-a6e8-c84756983ac7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:15:28.663091 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-26b3eee0-ce98-444a-9f7a-1abd440b5561 req-0fc91425-2e06-4bc6-a6e8-c84756983ac7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.1655183 May 24 10:15:28.717963 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-26b3eee0-ce98-444a-9f7a-1abd440b5561 req-19505412-825b-4afd-bb69-029b5e5e380c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475187 May 24 10:15:30.086672 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdd920ea-b12b-4f60-9bd8-7f7d81f046ef req-02655d23-173a-4cd2-a4c3-24b88c2fd78e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:30.089446 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-fdd920ea-b12b-4f60-9bd8-7f7d81f046ef req-02655d23-173a-4cd2-a4c3-24b88c2fd78e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.2557123 May 24 10:15:30.144206 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-fdd920ea-b12b-4f60-9bd8-7f7d81f046ef req-d8b406eb-1501-4489-8cbb-79d2d4618cb7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0483227 May 24 10:15:31.387752 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-37e203d5-c5aa-480b-91de-cc5db2902c28 req-5b1012ee-ae30-4480-9438-0befc4986d93 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:31.389445 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-37e203d5-c5aa-480b-91de-cc5db2902c28 req-5b1012ee-ae30-4480-9438-0befc4986d93 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.1018167 May 24 10:15:31.441275 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-37e203d5-c5aa-480b-91de-cc5db2902c28 req-b66b83e0-527a-4606-857f-77c62b7a7e7d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0451803 May 24 10:15:32.496667 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:15:32.673441 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec4d9286-436e-4440-bc3c-49c7428eb79e req-61d5a364-bea9-4f9b-a08e-72cc0462da8c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:32.674965 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ec4d9286-436e-4440-bc3c-49c7428eb79e req-61d5a364-bea9-4f9b-a08e-72cc0462da8c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.1111853 May 24 10:15:32.740505 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ec4d9286-436e-4440-bc3c-49c7428eb79e req-f4a1095e-57fc-451a-8bdf-8d52821b9a36 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0556970 May 24 10:15:33.346722 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-62ea6397-ac29-4225-9065-21aa68f680f3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:34.005880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-15035844-9489-4455-a8f1-1acd2ac6b70d req-687bb2f0-a17b-4511-9ab7-d9aec3040e39 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:34.007453 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-15035844-9489-4455-a8f1-1acd2ac6b70d req-687bb2f0-a17b-4511-9ab7-d9aec3040e39 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.1431046 May 24 10:15:34.058108 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-15035844-9489-4455-a8f1-1acd2ac6b70d req-eb5baf62-a4c5-4674-84eb-62634f574f3e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0440640 May 24 10:15:35.285719 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8b4bd12-3ad9-4791-9d05-52f48aef1d30 req-26400f57-9ae7-4d96-aac7-8d20dcb25ef9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:35.287243 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f8b4bd12-3ad9-4791-9d05-52f48aef1d30 req-26400f57-9ae7-4d96-aac7-8d20dcb25ef9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 944 time: 0.0830216 May 24 10:15:35.337396 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f8b4bd12-3ad9-4791-9d05-52f48aef1d30 req-18cf364d-4474-476a-b2c8-e96e8044f570 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0438924 May 24 10:15:35.809293 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-23bd0d00-ceaf-4e4a-b7ae-01fedd8e500e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:35.900921 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:36.286305 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 2faafcb9-9d9d-4bf2-997a-509be060bb30 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:15:36.430315 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:15:36.445572 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:15:36.504151 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:36.522782 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 2faafcb9-9d9d-4bf2-997a-509be060bb30 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:15:36.541111 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:15:36.609496 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:36.612320 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:36.695907 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:36.708370 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.708720 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.709085 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.709439 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.709773 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.710104 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.710510 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.710848 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.711194 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.711518 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.711850 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.712166 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:36.716196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:36.733184 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edfe6d0f-7c7d-42c5-87d3-389899656003 req-8d8f2246-7db5-43ef-83e3-ccb97aec2ec3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:36.734702 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edfe6d0f-7c7d-42c5-87d3-389899656003 req-8d8f2246-7db5-43ef-83e3-ccb97aec2ec3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 945 time: 0.2820563 May 24 10:15:36.784663 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-edfe6d0f-7c7d-42c5-87d3-389899656003 req-f97e16cd-cb8a-40f6-b16b-5d106ef6d5c6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0437543 May 24 10:15:36.865007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['2faafcb9-9d9d-4bf2-997a-509be060bb30'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:15:37.300512 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:37.994560 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-266f458e-0918-4849-a047-4e6345bb2bfc req-761f004d-96ef-45a2-915c-475cea3958c7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:37.996115 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-266f458e-0918-4849-a047-4e6345bb2bfc req-761f004d-96ef-45a2-915c-475cea3958c7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 945 time: 0.0896275 May 24 10:15:38.049731 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-266f458e-0918-4849-a047-4e6345bb2bfc req-836e1949-9665-4d77-b68e-7a14d5cd0cca tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0429542 May 24 10:15:38.279162 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 2faafcb9-9d9d-4bf2-997a-509be060bb30 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:15:38.415073 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port 2faafcb9-9d9d-4bf2-997a-509be060bb30 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:15:38.420181 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port 2faafcb9-9d9d-4bf2-997a-509be060bb30 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:15:38.420561 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:38.552850 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:38.596280 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:15:38.612541 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:15:38.656514 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'code': 200} May 24 10:15:38.723934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:38.727928 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-792ee7f4-d995-4c01-8fcd-d39ae9256a53 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:38.817985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:38.821766 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:38.872209 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:39.157265 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28ec5cb0-20c6-4e77-82e7-34450f663397 req-5d242ad8-6f0f-41bc-97ba-b1b3918d84b2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:39.159542 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-28ec5cb0-20c6-4e77-82e7-34450f663397 req-5d242ad8-6f0f-41bc-97ba-b1b3918d84b2 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.1076946 May 24 10:15:39.206210 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-28ec5cb0-20c6-4e77-82e7-34450f663397 req-1385fa6d-9fe3-4017-8434-5d6d32cebae3 service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0401661 May 24 10:15:39.282072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de1bdc1d-7205-4573-9126-c950a5e2fa0e req-f59334dc-d940-40c4-a6b0-4cb493aeaffd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:39.283578 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-de1bdc1d-7205-4573-9126-c950a5e2fa0e req-f59334dc-d940-40c4-a6b0-4cb493aeaffd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.0860355 May 24 10:15:39.341145 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-de1bdc1d-7205-4573-9126-c950a5e2fa0e req-4f7be3ec-0ced-47a4-810c-5ead669f571b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0515535 May 24 10:15:40.542629 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c64f0194-f3b8-4d0b-8928-9840b55927a0 req-70403446-9464-49e3-bdb5-09f40efdaaa9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:40.544289 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c64f0194-f3b8-4d0b-8928-9840b55927a0 req-70403446-9464-49e3-bdb5-09f40efdaaa9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.0900323 May 24 10:15:40.600410 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c64f0194-f3b8-4d0b-8928-9840b55927a0 req-d0bff59d-478f-4e2d-b978-dfcc327dfa90 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0458796 May 24 10:15:40.851127 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b579abe4-1f5e-454b-931a-42fcd5c8c61d req-05d5444d-3990-4064-b25a-90ff580c92c0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:40.853029 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b579abe4-1f5e-454b-931a-42fcd5c8c61d req-05d5444d-3990-4064-b25a-90ff580c92c0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.0878079 May 24 10:15:40.902393 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b579abe4-1f5e-454b-931a-42fcd5c8c61d req-aac9b5e2-c4f9-4061-ad39-699051a13e69 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0426900 May 24 10:15:40.928042 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34426) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:15:41.009341 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a86665e0-cac3-4ec7-9973-3c89832c3555 tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:41.010878 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a86665e0-cac3-4ec7-9973-3c89832c3555 tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1094 time: 0.0818300 May 24 10:15:41.015737 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34430) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:15:41.021227 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address': '172.20.0.12'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:15:41.027730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:15:41.032649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:floatingip and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:15:41.035732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for floatingip was recalculated. Used quota:1. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:15:41.042004 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource floatingip. Total usage: 1; quota limit: 50; headroom:49 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:15:41.100606 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:41.251848 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:41.398808 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.27.0.220 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/f0501b6b-549c-4b36-8506-eacc6c2f511f) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:15:41.656763 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:41.769835 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:41.773991 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-6efce7d5-4a6e-4578-b268-29f4eec33661 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:41.849913 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:15:41.853145 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:15:41.856386 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:15:41.892729 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=f0501b6b-549c-4b36-8506-eacc6c2f511f,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:41.912399 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.912793 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.913195 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.913571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.913932 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.914295 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.914658 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.915020 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.915377 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.915733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.916215 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.916437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:15:41.922711 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:42.338568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:42.474252 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:42.703826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:42.781613 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:42.786015 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5327c031-a90e-4d73-a7e0-0941fbca1653 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:15:42.901335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=f0501b6b-549c-4b36-8506-eacc6c2f511f,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:15:42.906865 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:15:42.917364 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:43.100371 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 943c5695-b38a-44fd-9bb5-a0a4de803e89 associated. External IP: 172.27.0.220, port: 2faafcb9-9d9d-4bf2-997a-509be060bb30. May 24 10:15:43.684126 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:15:43.886372 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:15:43.900552 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:15:43.911939 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:15:43.912605 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:15:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:44.055647 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:44.158833 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 997 time: 3.1415529 May 24 10:15:44.159685 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:15:44.176624 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:15:44.244204 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'status': 'completed', 'code': 200} May 24 10:15:44.344926 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-8ee2a294-3add-40fc-8389-739b62b7a11e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:44.346491 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-8ee2a294-3add-40fc-8389-739b62b7a11e service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.0819001 May 24 10:15:44.496853 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-aa0ee73c-dd05-447f-85d3-f4b038f3d324 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:44.498931 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-aa0ee73c-dd05-447f-85d3-f4b038f3d324 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1465337 May 24 10:15:44.611324 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-4fe05ef8-a16e-43cc-a223-a7bcc1d109f7 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 1000 time: 0.1053913 May 24 10:15:44.680534 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-ed56c157-50dd-43ea-b187-a9e98ac1144c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:44.682044 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-ed56c157-50dd-43ea-b187-a9e98ac1144c service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0652113 May 24 10:15:44.697793 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:42Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:44.705895 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:15:44.722133 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:15:44.723219 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:15:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:44.763419 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-ffad1771-5b38-4fab-87db-59c0dd93c206 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:44.765419 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-ffad1771-5b38-4fab-87db-59c0dd93c206 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0768855 May 24 10:15:44.928767 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-3f783d5c-a082-4626-bfa5-058d2541a14d service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1569517 May 24 10:15:44.997883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:15:45.093803 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-1465937a-5389-4f46-88ef-a8879d19cfa8 req-f9206dfa-ca25-48a9-b5ac-6efcd15b564e service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1589310 May 24 10:15:45.397855 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:42Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:15:45.405791 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-84bc564a-4047-40f8-933d-86b2a39cedd3 None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:42Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:15:45.619924 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-af338f70-927f-4fb3-a1ad-08ba5839b4f7 None None] New status for floating IP 943c5695-b38a-44fd-9bb5-a0a4de803e89: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:15:50.334813 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:15:56.519616 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-17e7ea24-c86f-4a9e-8790-904b10869ca8 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1416531 May 24 10:15:56.687955 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-61e5da4e-6996-44b7-b18b-dc428f35e8eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:56.689991 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-61e5da4e-6996-44b7-b18b-dc428f35e8eb service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.0917459 May 24 10:15:56.866332 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-79fab432-3a59-40ac-8382-891fddbb55bd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:56.868661 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-79fab432-3a59-40ac-8382-891fddbb55bd service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1536522 May 24 10:15:56.995665 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-50d0d9f1-8f14-4237-9478-6e38468bd301 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 1002 time: 0.1191387 May 24 10:15:57.070978 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-9e126a5b-46ca-474f-8bbc-a3ce585ebe8a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:57.072565 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-9e126a5b-46ca-474f-8bbc-a3ce585ebe8a service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0696440 May 24 10:15:57.164085 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-9777fb72-e36b-4516-88db-763b56f4cdb6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:15:57.166649 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-9777fb72-e36b-4516-88db-763b56f4cdb6 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0858815 May 24 10:15:57.438979 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-6f62bb8e-7a17-4bf1-a788-71f289069919 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2648313 May 24 10:15:57.585392 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-08f969b1-00cb-437a-8cf0-5359ee9f2faa req-97a646d2-e3a1-441e-a6be-b51a78b78d51 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1390619 May 24 10:16:03.323154 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-1fb4d92e-a29f-412c-b3d8-a0570849f6c9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:05.831424 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-7b10f3f0-159a-46a3-a745-667a5458a7e7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:05.939406 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:07.316993 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:08.842710 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:09.497274 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:16:20.259569 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:33.356642 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-91accb2a-a9c6-4924-945c-8238501d5e6b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:35.830069 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-26e77314-b3da-4242-bc9f-083da1a67e95 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:35.938727 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:37.281797 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:38.836063 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:46.495894 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:16:50.293803 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:16:57.448015 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-04033d91-d671-4865-8d52-34a25b39e0ff service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0753987 May 24 10:16:57.606954 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-2107c8b5-7e10-4532-8dcc-eab34fadf669 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:16:57.608431 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-2107c8b5-7e10-4532-8dcc-eab34fadf669 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.0825257 May 24 10:16:57.771233 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-42be7438-4a5f-40b5-810d-bc9ed006989f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:16:57.772803 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-42be7438-4a5f-40b5-810d-bc9ed006989f service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1401644 May 24 10:16:57.885571 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-7a08ebe3-1d8a-481f-8895-62e4d54199c2 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 1002 time: 0.1049988 May 24 10:16:57.952579 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-f7c26807-6075-46d4-afb0-6b93591c4796 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:16:57.954122 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-f7c26807-6075-46d4-afb0-6b93591c4796 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0622647 May 24 10:16:58.044884 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-90c76699-b8fb-44ad-9a76-82b58c586094 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:16:58.046373 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-90c76699-b8fb-44ad-9a76-82b58c586094 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0826004 May 24 10:16:58.193510 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-52ef108e-3c29-4566-94dc-2aa1729a5bbe service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1404536 May 24 10:16:58.345912 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-cbd37123-73f4-485a-a1d7-0d18ae765f5b req-d3fac02f-1b11-4d24-be83-d974367494ef service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1449525 May 24 10:17:03.332300 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-5bbe3cab-cb7a-423e-bcd8-26985047fbf6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:05.869845 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-57d7e730-7953-4217-bbd3-8cc2cbdf7094 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:05.938743 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:07.293080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:08.840944 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:15.344465 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-22d4c906-2b9b-4b2f-b9f5-68afd33e62b6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-525824861&fields=id&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 267 time: 0.0463066 May 24 10:17:15.518259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-335d7e1e-f931-428f-8898-5dfb095428bf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:15.519015 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-335d7e1e-f931-428f-8898-5dfb095428bf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:network {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:17:15.635623 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-335d7e1e-f931-428f-8898-5dfb095428bf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:network {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:17:15.637527 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-335d7e1e-f931-428f-8898-5dfb095428bf tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.2815366 May 24 10:17:15.655381 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-6c29545c-6a65-4eb4-8b38-cb8f825a0b13 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/quotas/2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 375 time: 0.0117571 May 24 10:17:15.755490 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-283ea2dd-bfce-439f-afee-bdde187e7c56 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&fields=id HTTP/1.1" status: 200 len: 440 time: 0.0938935 May 24 10:17:15.818043 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-585454f3-05b9-4f56-bfe2-9ef46e63f3e3 service neutron] http://10.222.0.67:9696/v2.0/extensions returned with HTTP 200 {{(pid=109112) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 10:17:15.819096 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-585454f3-05b9-4f56-bfe2-9ef46e63f3e3 service neutron] 10.222.0.67 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0506551 May 24 10:17:15.982201 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c36bffaf-d255-40d6-99b5-b2ba14392763 service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1514063 May 24 10:17:16.133300 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-97f71fbb-ee37-4a25-9b51-f111816b3adb service neutron] 10.222.0.67 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1404252 May 24 10:17:16.909614 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ab8708dc-dd58-4b13-9ad5-3003fd8ab840 req-f0909982-5aa6-4e0e-a4d8-b095c8af5861 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.0563743 May 24 10:17:18.108117 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-d7334368-70a1-4a5d-9774-ecad70987ab8 req-6238a247-61da-431e-bb97-20864c9065db tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.0527754 May 24 10:17:19.285804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-5131c46b-6139-4c87-9d59-2667722cb377 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:19.287405 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-5131c46b-6139-4c87-9d59-2667722cb377 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 812 time: 0.1811390 May 24 10:17:19.390241 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-5ee9d248-31ad-4cce-a11f-d4b26be882ae req-856c20c2-ce35-448e-8b7e-86a308e7a0db tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.0612364 May 24 10:17:19.400973 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-e9193372-3dc2-4dad-9725-a3eddf362cb0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52 HTTP/1.1" status: 200 len: 379 time: 0.0997970 May 24 10:17:19.415276 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'port': {'device_id': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'admin_state_up': True, 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'security_groups': ['2b6d1377-7673-408e-86ee-8125745d3311']}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:17:19.638889 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:17:19.653579 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.quota.driver [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource port. Total usage: 4; quota limit: 500; headroom:496 {{(pid=109113) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:17:19.829514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Made reservation on behalf of 2cc9bd0e4be64b0e9add9a9f9e8bbe52 for: {'port': 1} {{(pid=109113) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 10:17:19.833267 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:20.062527 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.db_base_plugin_common [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.20.0.9 (3670d884-4bc4-4d4e-8dbb-1165ed224ea7/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51/f1ec19f0-7419-4db3-9689-1d76c1528de8) {{(pid=109113) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:17:20.372463 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:20.410600 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:20.431409 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Transition to ACTIVE for port object f1ec19f0-7419-4db3-9689-1d76c1528de8 will not be triggered until provisioned by entity DHCP. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:17:20.590003 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-f21d11ec-e49a-4043-b9ef-232eb740d2ba req-a10ba76d-4370-4b7d-b9d2-a60fdf6cd87d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.0580254 May 24 10:17:20.658587 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_create {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:20.669002 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b674f891-bece-4aa1-80a4-886fb5364736 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:20.736373 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 is already hosted by enough agents. {{(pid=109113) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:17:20.739788 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.base_scheduler [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109113) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:17:20.744518 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109113) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:17:20.789646 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=1']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:20.815883 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.816389 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.816896 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.817421 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.817891 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.818502 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.819010 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.821223 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.822018 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.822664 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.823880 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.824408 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:20.831804 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.167s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:20.864861 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:20.866054 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:21.001297 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c5dfae9f-12ad-4750-ad11-4aa12f60f12d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.5935323 May 24 10:17:21.014604 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.v2.base [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Request body: {'port': {'device_id': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-79123'}} {{(pid=109113) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:17:21.143392 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:21.172449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-fb761aeb-27cd-4dff-b8c7-a39109767aad None None] Provisioning for port f1ec19f0-7419-4db3-9689-1d76c1528de8 completed by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:17:21.181618 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-fb761aeb-27cd-4dff-b8c7-a39109767aad None None] Provisioning complete for port f1ec19f0-7419-4db3-9689-1d76c1528de8 triggered by entity DHCP. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:17:21.182206 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fb761aeb-27cd-4dff-b8c7-a39109767aad None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:21.288871 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-fb761aeb-27cd-4dff-b8c7-a39109767aad None None] Port f1ec19f0-7419-4db3-9689-1d76c1528de8 cannot update to ACTIVE because it is not bound. {{(pid=109114) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 10:17:21.554085 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f1ec19f0-7419-4db3-9689-1d76c1528de8 {{(pid=109113) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:17:21.562337 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks [] for port, precommit_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:21.581994 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.provisioning_blocks [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Transition to ACTIVE for port object f1ec19f0-7419-4db3-9689-1d76c1528de8 will not be triggered until provisioned by entity L2. {{(pid=109113) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 10:17:21.640436 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:21.646868 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7b50b704-b405-45fb-b9de-ccca0a5f0e68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:21.834330 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Pushing event updated for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=2']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:21.852717 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:21.988977 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75c4e868-453a-4e2a-bc8a-74d33e59894b req-969ee80b-c0a8-4509-be1b-5b7098d38398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:21.989761 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-75c4e868-453a-4e2a-bc8a-74d33e59894b req-969ee80b-c0a8-4509-be1b-5b7098d38398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:17:21.992819 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Attempting to bind port f1ec19f0-7419-4db3-9689-1d76c1528de8 on host hv-cinder-79123 for vnic_type normal with profile {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 10:17:21.993186 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Attempting to bind port f1ec19f0-7419-4db3-9689-1d76c1528de8 by drivers openvswitch,hyperv on host hv-cinder-79123 at level 0 using segments [{'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 10:17:21.993514 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Attempting to bind port f1ec19f0-7419-4db3-9689-1d76c1528de8 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:17:22.004057 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Port f1ec19f0-7419-4db3-9689-1d76c1528de8 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-79123 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 10:17:22.004361 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Attempting to bind port f1ec19f0-7419-4db3-9689-1d76c1528de8 on network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 10:17:22.016608 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Checking agent: {'id': '1b0cbdfb-4220-4151-a687-0c4b4be07761', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-79123', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'started_at': datetime.datetime(2021, 5, 24, 9, 58, 50), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 10, 17, 20), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 10:17:22.016945 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Checking segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=109113) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 10:17:22.017364 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Bound using segment: {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'} {{(pid=109113) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 10:17:22.017807 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.managers [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Bound port: f1ec19f0-7419-4db3-9689-1d76c1528de8, host: hv-cinder-79123, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 82, 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7'}}] {{(pid=109113) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 10:17:22.018154 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:22.111801 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-75c4e868-453a-4e2a-bc8a-74d33e59894b req-969ee80b-c0a8-4509-be1b-5b7098d38398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:17:22.113261 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-75c4e868-453a-4e2a-bc8a-74d33e59894b req-969ee80b-c0a8-4509-be1b-5b7098d38398 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.3566468 May 24 10:17:22.188625 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, cleared binding levels {{(pid=109113) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 10:17:22.193370 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-75c4e868-453a-4e2a-bc8a-74d33e59894b req-6c36229e-d0a0-472f-8c2d-2191cbd2186e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0724144 May 24 10:17:22.200393 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 10:17:22.293529 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129003724'] for port, after_update {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:22.298411 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-49926b71-251b-4da1-8654-943a28130f70 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:22.369744 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:22.370442 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:22.371883 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] 10.222.0.95 "PUT /v2.0/ports/f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 1093 time: 1.3627582 May 24 10:17:22.372446 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:17:22.402225 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Pushing event updated for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=3']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:22.406399 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-52992e9e-43fa-40af-ac18-3cf29759c7c7 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:22.553202 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'status': 'completed', 'code': 200} May 24 10:17:22.611568 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-551db170-59d5-4699-9e08-cb7807920f7c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:22.614251 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-551db170-59d5-4699-9e08-cb7807920f7c service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1096 time: 0.1116784 May 24 10:17:22.702709 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-c6eea60e-3b65-4435-aa6c-b59b8257bb9b service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 217 time: 0.0771346 May 24 10:17:22.772365 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-01cbe44a-978e-40c8-8493-bae98602bf03 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:22.774041 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-01cbe44a-978e-40c8-8493-bae98602bf03 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0648115 May 24 10:17:22.878052 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-9c277e2c-c896-46c5-a8d0-88d169da0bdf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:22.879850 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-9c277e2c-c896-46c5-a8d0-88d169da0bdf service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0952749 May 24 10:17:23.144975 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-b49a1cdc-fd5d-43ca-8b6e-01c97de91378 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2577145 May 24 10:17:23.299167 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4f0192a9-bbfd-4dfc-9bf1-f087c3a6ffd4 req-ca4e9543-3874-4a68-a8d9-b0d2e89762bc service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1474860 May 24 10:17:23.497882 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:17:23.543136 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0ecbd52-8089-49cb-a218-77a00b9f5b46 req-17546292-5db2-4b88-b5b0-462477214458 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:23.545003 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0ecbd52-8089-49cb-a218-77a00b9f5b46 req-17546292-5db2-4b88-b5b0-462477214458 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.1613965 May 24 10:17:23.547342 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8067852-aadd-4040-91b5-e60e72694a06 req-8daae294-06e1-4196-a360-db632da9c470 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:23.548990 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-8daae294-06e1-4196-a360-db632da9c470 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1096 time: 0.1297255 May 24 10:17:23.684294 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f0ecbd52-8089-49cb-a218-77a00b9f5b46 req-e5232355-753d-48a8-b4c6-8affd0b3f60b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1328740 May 24 10:17:23.738661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8067852-aadd-4040-91b5-e60e72694a06 req-31d4e5f1-1d6f-4772-8986-078e4a17fc0e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:23.740192 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-31d4e5f1-1d6f-4772-8986-078e4a17fc0e service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1749706 May 24 10:17:23.827593 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-6e0cd2ce-3456-4494-af03-1df594f76f4d service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 217 time: 0.0709920 May 24 10:17:23.903390 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8067852-aadd-4040-91b5-e60e72694a06 req-7f89f8da-e527-432b-926a-f26410402844 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:23.904925 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-7f89f8da-e527-432b-926a-f26410402844 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0621014 May 24 10:17:23.990651 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8067852-aadd-4040-91b5-e60e72694a06 req-eb935284-04c2-4f31-b9f2-ec911e40ece7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:23.992421 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-eb935284-04c2-4f31-b9f2-ec911e40ece7 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0796885 May 24 10:17:24.136644 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-4787b8ce-09bf-461f-a44b-62fcddf6e0a8 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1370704 May 24 10:17:24.288330 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-f8067852-aadd-4040-91b5-e60e72694a06 req-04b62a6c-f784-4413-8f41-fda4ec58f462 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1450522 May 24 10:17:24.889846 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00c42754-a883-4e3d-acbe-cc329f794f8b req-696c5bf4-1380-455f-ac34-0a3a2d9a1e7e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:24.891473 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-00c42754-a883-4e3d-acbe-cc329f794f8b req-696c5bf4-1380-455f-ac34-0a3a2d9a1e7e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.0893538 May 24 10:17:24.949935 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-00c42754-a883-4e3d-acbe-cc329f794f8b req-9c3dbb02-2e41-401b-9720-8d58a5fe0de4 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475185 May 24 10:17:26.156310 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac3a67b9-d498-43cf-9380-a005190762d2 req-fc738bd2-d008-400c-ad0b-a572699039d3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:26.157854 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ac3a67b9-d498-43cf-9380-a005190762d2 req-fc738bd2-d008-400c-ad0b-a572699039d3 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.0918279 May 24 10:17:26.205570 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-ac3a67b9-d498-43cf-9380-a005190762d2 req-939011d5-295b-4a17-a2c6-88b0f73ce094 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0417864 May 24 10:17:27.443106 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7fbce21-0470-4ad6-a2ca-5ff09697a48e req-66b4240a-6ee4-4a99-abef-a7f27e3c1190 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:27.445803 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c7fbce21-0470-4ad6-a2ca-5ff09697a48e req-66b4240a-6ee4-4a99-abef-a7f27e3c1190 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.1027460 May 24 10:17:27.530609 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c7fbce21-0470-4ad6-a2ca-5ff09697a48e req-7ca52aaf-6bcb-4405-bdb0-345233291652 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0783892 May 24 10:17:28.736114 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45d13cbf-bef3-4259-81be-03cc18bc85b0 req-1f30e173-9bf3-4da1-851e-7645564838d0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:28.737736 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45d13cbf-bef3-4259-81be-03cc18bc85b0 req-1f30e173-9bf3-4da1-851e-7645564838d0 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.0897338 May 24 10:17:28.796513 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-45d13cbf-bef3-4259-81be-03cc18bc85b0 req-099d69ca-963c-4352-a68e-40b0bc285522 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0486794 May 24 10:17:30.050436 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f7dfb48-fa46-40ef-913f-4f755217584a req-bf57c695-c2e5-41f7-a7b9-05257d1c39c1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:30.052038 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6f7dfb48-fa46-40ef-913f-4f755217584a req-bf57c695-c2e5-41f7-a7b9-05257d1c39c1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 943 time: 0.0912967 May 24 10:17:30.100969 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-6f7dfb48-fa46-40ef-913f-4f755217584a req-c1270141-979c-4e1d-b1f7-60b275bf0e4a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0423415 May 24 10:17:30.728795 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f1ec19f0-7419-4db3-9689-1d76c1528de8 details requested by agent hyperv_hv-cinder-79123 with host hv-cinder-79123 {{(pid=109114) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 10:17:30.949902 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:17:30.965656 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Returning: {'device': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'port_id': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'mac_address': 'fa:16:3e:56:a8:d9', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 82, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=109114) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 10:17:31.025107 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:31.058395 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f1ec19f0-7419-4db3-9689-1d76c1528de8 {{(pid=109114) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 10:17:31.086913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:17:31.178247 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:31.180781 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-2e232918-9cf0-44df-a5c4-c3040019b133 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:31.260224 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=4']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:31.268319 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.268685 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.269118 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.269452 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.269777 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.270141 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.270467 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.270814 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.271136 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.271453 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.271771 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.272084 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:31.276972 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:31.321126 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ae46648-8d91-418c-962f-a64e193a3984 req-9d533908-9e56-4cd4-8fca-30a58824c542 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:31.323275 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4ae46648-8d91-418c-962f-a64e193a3984 req-9d533908-9e56-4cd4-8fca-30a58824c542 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 944 time: 0.1021259 May 24 10:17:31.379276 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4ae46648-8d91-418c-962f-a64e193a3984 req-6d673f3c-6a46-4afc-b432-7cd4ad14db58 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0490999 May 24 10:17:31.419894 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] get_ports_and_sgs() called for port_ids ['f1ec19f0-7419-4db3-9689-1d76c1528de8'] {{(pid=109114) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 10:17:32.639139 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ad08701-02b2-4103-8147-f62a2f00034a req-77ce65f0-cc90-4f84-89a7-ecc6a6d28b57 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:32.640608 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2ad08701-02b2-4103-8147-f62a2f00034a req-77ce65f0-cc90-4f84-89a7-ecc6a6d28b57 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 944 time: 0.0965650 May 24 10:17:32.695551 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-2ad08701-02b2-4103-8147-f62a2f00034a req-d0415945-d204-43ef-a34f-be9ecce6ea99 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0487349 May 24 10:17:32.728148 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f1ec19f0-7419-4db3-9689-1d76c1528de8 up at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 10:17:32.897863 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning for port f1ec19f0-7419-4db3-9689-1d76c1528de8 completed by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 10:17:32.903818 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.provisioning_blocks [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Provisioning complete for port f1ec19f0-7419-4db3-9689-1d76c1528de8 triggered by entity L2. {{(pid=109114) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 10:17:32.904256 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-3633966'] for port, provisioning_complete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:33.032961 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:33.068534 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8'}] {{(pid=109114) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:17:33.082109 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:17:33.120742 dv-cinder-791239-2-iscsi neutron-server[109114]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'code': 200} May 24 10:17:33.155435 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287'] for port, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:33.158444 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:33.300406 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Pushing event updated for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=5']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:33.305672 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:33.491643 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-36614314-677d-478e-a709-2668243dc50b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:33.647633 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-246b6fdf-fb48-4d43-9a67-5d38df02128c req-e7b49a00-1adc-43b8-944f-a62364375e63 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:33.649481 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-246b6fdf-fb48-4d43-9a67-5d38df02128c req-e7b49a00-1adc-43b8-944f-a62364375e63 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1098 time: 0.0863972 May 24 10:17:33.703239 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-246b6fdf-fb48-4d43-9a67-5d38df02128c req-44e933eb-93fe-4c5a-b947-3f56b592a3de service neutron] 10.222.0.95 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0469143 May 24 10:17:33.920532 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7cbbd5aa-37a7-4c2e-8f8f-0b5c012a8111 req-357a1e65-3adb-481e-bec0-e765ae258e97 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:33.922539 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7cbbd5aa-37a7-4c2e-8f8f-0b5c012a8111 req-357a1e65-3adb-481e-bec0-e765ae258e97 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.0981064 May 24 10:17:33.974145 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-7cbbd5aa-37a7-4c2e-8f8f-0b5c012a8111 req-80ad6e1a-cd31-4f69-a0d0-8f7fb668e9a6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441508 May 24 10:17:35.187859 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f0a128d-91ba-4cab-ab7d-67663e4f8c6c req-b0c439b1-fecd-45e5-9bff-eb090648ae0b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:35.189374 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f0a128d-91ba-4cab-ab7d-67663e4f8c6c req-b0c439b1-fecd-45e5-9bff-eb090648ae0b tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.0867226 May 24 10:17:35.251120 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f0a128d-91ba-4cab-ab7d-67663e4f8c6c req-d961b5d6-017f-41a5-b4f0-dc18a49cafa4 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0555723 May 24 10:17:35.516569 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b90dd146-b6ce-464c-8256-763175260183 req-f2c26a98-8049-4996-86d6-2e17725bcd3a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:35.519585 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b90dd146-b6ce-464c-8256-763175260183 req-f2c26a98-8049-4996-86d6-2e17725bcd3a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.1530285 May 24 10:17:35.608637 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-b90dd146-b6ce-464c-8256-763175260183 req-e97ee973-80ea-4fc9-afd8-b370c79a8a4c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0826921 May 24 10:17:35.637687 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34558) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:17:35.827446 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-f2007726-d3aa-44a1-a485-ad48c23e6314 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:35.833469 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-65a8bfd7-e7e3-4ad8-8ce1-15c2da409aba tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:35.836957 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-65a8bfd7-e7e3-4ad8-8ce1-15c2da409aba tempest-TestVolumeBootPattern-691055854 tempest-TestVolumeBootPattern-691055854-project-admin] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1093 time: 0.1971068 May 24 10:17:35.840433 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34562) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:17:35.846259 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.v2.base [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Request body: {'floatingip': {'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'port_id': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address': '172.20.0.9'}} {{(pid=109112) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 10:17:35.853337 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 10:17:35.858566 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Usage tracker for resource:floatingip and tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 is out of sync, need to count used quota {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 10:17:35.862327 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Quota usage for floatingip was recalculated. Used quota:2. {{(pid=109112) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 10:17:35.870247 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.quota.driver [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attempting to reserve 1 items for resource floatingip. Total usage: 2; quota limit: 50; headroom:48 {{(pid=109112) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 10:17:35.880533 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:35.966432 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:36.130251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:36.285649 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_common [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Allocated IP 172.27.0.189 (2211f181-5fdc-4580-9f8a-8610ff86d56a/c29e3891-6552-402c-9c30-4a0d6fd7e6a6/230f54ea-3cad-4379-9dc0-54f9a046699d) {{(pid=109112) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 10:17:36.456571 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:36.554730 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8762070492380', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:36.558892 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-46410bf5-d82b-4c01-9788-ea0809501e3b tempest-AttachVolumeTestJSON-882843242 tempest-AttachVolumeTestJSON-882843242-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:36.630006 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Network 2211f181-5fdc-4580-9f8a-8610ff86d56a is already hosted by enough agents. {{(pid=109112) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 10:17:36.633014 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.base_scheduler [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Resources bound (agent type, host, resource id): {{(pid=109112) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 10:17:36.636631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Received port after_create {{(pid=109112) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 10:17:36.679012 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=230f54ea-3cad-4379-9dc0-54f9a046699d,revision_number=1']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:36.697366 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.698342 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.698732 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.699136 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.699507 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.699876 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.700246 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.700611 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.700990 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.701382 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.701744 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.702102 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:17:36.707930 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:37.131491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_create {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:37.280867 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:37.316741 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-3570200'] for port, before_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:37.543707 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for port, precommit_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:37.617024 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8762070893460', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8762070492398', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8762070496287', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-127970283'] for port, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:37.621788 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7931996b-9c17-4494-883a-f11bca77853e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:17:37.741612 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event updated for resources: {'Port': ['ID=230f54ea-3cad-4379-9dc0-54f9a046699d,revision_number=2']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:17:37.748367 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:17:37.758475 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-425282'] for floatingip, after_update {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:38.055060 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 7df5eb14-1837-465a-8434-005a711bdbcc associated. External IP: 172.27.0.189, port: f1ec19f0-7419-4db3-9689-1d76c1528de8. May 24 10:17:38.443915 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:17:38.638358 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:17:38.652075 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:17:38.668464 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:17:38.669567 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:17:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:17:38.783232 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:38.832968 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:38.865086 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-92c8ae6f-5142-428c-9ed7-9e4fa38f3289 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.0118942 May 24 10:17:38.865086 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:17:38.882239 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:17:39.013693 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'status': 'completed', 'code': 200} May 24 10:17:39.249677 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-f2c73679-5043-4c75-bfaa-2d4eb2492de1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:39.252276 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-f2c73679-5043-4c75-bfaa-2d4eb2492de1 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1098 time: 0.2114325 May 24 10:17:39.399167 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '7df5eb14-1837-465a-8434-005a711bdbcc', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.189', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:56:a8:d9', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:17:37Z', 'updated_at': '2021-05-24T10:17:37Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}, {'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:17:39.408196 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:17:39.424752 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:17:39.425834 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:17:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:17:39.475767 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-ec89f9df-e12c-44df-abd5-13fb60a68ee1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:39.478452 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-ec89f9df-e12c-44df-abd5-13fb60a68ee1 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.2176576 May 24 10:17:39.619065 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-cf3c7634-0fee-4e63-863b-3a021808dea8 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 999 time: 0.1333210 May 24 10:17:39.663509 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:17:39.710775 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-3d9d035a-e45c-4558-aea6-fa05c2ada627 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:39.713496 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-3d9d035a-e45c-4558-aea6-fa05c2ada627 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0760841 May 24 10:17:39.810450 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-831bc4fe-4b91-419d-b459-1fa9aee220e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:39.812118 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-831bc4fe-4b91-419d-b459-1fa9aee220e1 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0894282 May 24 10:17:40.042704 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-75b19f64-ef40-4f79-8f5c-d60da0dd4bb7 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2230055 May 24 10:17:40.162229 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '7df5eb14-1837-465a-8434-005a711bdbcc', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.189', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:56:a8:d9', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:17:37Z', 'updated_at': '2021-05-24T10:17:37Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}, {'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:17:40.167872 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5dd287ea-b3e7-43ec-b964-596faf3f704d None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '7df5eb14-1837-465a-8434-005a711bdbcc', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.189', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:56:a8:d9', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:17:37Z', 'updated_at': '2021-05-24T10:17:37Z', 'revision_number': 0, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}, {'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:17:40.200334 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-4c2619ac-57f1-4687-a0ed-a8d178e7df84 req-7c06c061-3612-4307-b553-d4b17fc06649 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1513350 May 24 10:17:40.384680 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fb096378-76d8-47ad-b0dd-0a0b88e68901 None None] New status for floating IP 7df5eb14-1837-465a-8434-005a711bdbcc: ACTIVE {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:17:50.365433 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:17:59.269857 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [req-7939995c-30b0-4e6f-8728-642df43d147f req-cd194f83-64d9-4ac5-9f89-cc5ba9282b01 service neutron] (109113) timed out ('10.222.0.67', 55884) {{(pid=109113) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 10:17:59.446915 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-ed1ef2f0-91cf-4bd1-ad04-edf677f4c410 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0778756 May 24 10:17:59.602991 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-2fb3f46b-71e8-408c-a2cc-71c673e2dab7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:59.605116 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-2fb3f46b-71e8-408c-a2cc-71c673e2dab7 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.0829089 May 24 10:17:59.764893 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-1963b3eb-f749-47c9-9bde-dc91bc66321e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:59.766459 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-1963b3eb-f749-47c9-9bde-dc91bc66321e service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1382513 May 24 10:17:59.877961 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-fc3f1cd9-4c4e-4a98-a6b9-b3ba3415e0d4 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 1002 time: 0.1035051 May 24 10:17:59.953068 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-6319547a-de18-46b7-817f-a4c132453649 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:17:59.954728 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-6319547a-de18-46b7-817f-a4c132453649 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0665617 May 24 10:18:00.037501 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-11ecabda-f2b2-4c3b-8101-7a364b46b8ef service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:18:00.039016 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-11ecabda-f2b2-4c3b-8101-7a364b46b8ef service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0765293 May 24 10:18:00.200440 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-5669c11b-6ff9-4780-85ba-5234ddb89205 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1465387 May 24 10:18:00.344865 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-fdfa643e-854b-43d2-b5a8-ae65a89f5bea req-16543568-5aa9-4152-9d50-1d00dfc0c935 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1375391 May 24 10:18:00.498212 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:18:03.336768 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-681404a6-c460-41cd-9af7-c384f0a0ab61 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:05.897447 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-83f8d75c-a8e8-4b0f-8249-5747be84c347 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:05.924950 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:07.376390 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:08.827912 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:20.319001 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:33.353228 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-dbc3caa1-ede6-4918-ba2c-89645aae611d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:35.845175 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-0e7e0e17-35d5-4823-b638-3ed223ffbb17 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:35.902058 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:37.272550 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:37.497377 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:18:38.822295 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:50.318092 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:18:59.593458 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-0afd0246-0570-4154-8619-26aff4294e06 service neutron] 10.222.0.95 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2039602 May 24 10:18:59.697117 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-9f1d86e9-3ede-4489-bafa-007b386e136d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:18:59.698657 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-9f1d86e9-3ede-4489-bafa-007b386e136d service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1098 time: 0.0823982 May 24 10:18:59.966491 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-c9034f76-6813-4a64-9e1a-308cc71582fb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:18:59.968854 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-c9034f76-6813-4a64-9e1a-308cc71582fb service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.2424505 May 24 10:19:00.090327 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-64cfc47b-434a-4234-9618-c5a2eee432c1 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 1001 time: 0.1144226 May 24 10:19:00.184182 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-2a586b95-9f5b-49cd-8488-b71722fb7d7a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:00.185743 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-2a586b95-9f5b-49cd-8488-b71722fb7d7a service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0890565 May 24 10:19:00.273402 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-5342b24d-1f1c-44da-86d1-b6edde154d59 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:00.275627 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-5342b24d-1f1c-44da-86d1-b6edde154d59 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0826402 May 24 10:19:00.423213 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-dae79d05-d9f8-4978-b42c-9b06e056f465 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1379449 May 24 10:19:00.568033 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-48a0f753-9ff3-4e84-ac38-8ac78858e677 req-6a785c42-d3d1-4b48-b5c2-5026d72871cd service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1374273 May 24 10:19:03.323784 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b8561389-686d-4722-ab25-34254639feb0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:05.820847 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9fe14bba-48ab-4b2f-af9f-7b2878bd510c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:05.935044 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:07.295029 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:08.860934 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:11.769260 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34594) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:12.090491 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port 230f54ea-3cad-4379-9dc0-54f9a046699d {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:12.090822 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:12.243768 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:12.244077 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:12.381022 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for 230f54ea-3cad-4379-9dc0-54f9a046699d owned by network:floatingip {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:12.575691 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:12.580659 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-04b9a851-900e-413c-bd23-dcc46bef2ca1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:12.629963 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=230f54ea-3cad-4379-9dc0-54f9a046699d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:12.674501 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.675037 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.675589 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.676137 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.676610 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.677116 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.677587 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.678018 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.678434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.678901 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.680038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.680038 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109112) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:12.693843 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:12.885352 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:12.886165 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.db.l3_db [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 7df5eb14-1837-465a-8434-005a711bdbcc disassociated (deleted). External IP: 172.27.0.189, port: f1ec19f0-7419-4db3-9689-1d76c1528de8. May 24 10:19:13.290720 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:19:13.490531 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:19:13.505202 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:19:13.516522 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:13.517395 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:13.609958 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:13.703153 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-44a5d1ce-37b5-47ac-85b9-9e8694e60ba6 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "DELETE /v2.0/floatingips/7df5eb14-1837-465a-8434-005a711bdbcc HTTP/1.1" status: 204 len: 168 time: 1.9323893 May 24 10:19:13.704683 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8'}] {{(pid=109112) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:19:13.725592 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:13.793204 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'status': 'completed', 'code': 200} May 24 10:19:13.956770 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-576367ba-7e43-41c5-810a-4ee191118670 req-9bb9d223-8ab9-4956-9b1c-179cba3e6580 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:13.959356 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-9bb9d223-8ab9-4956-9b1c-179cba3e6580 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 1098 time: 0.1379552 May 24 10:19:14.183153 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-576367ba-7e43-41c5-810a-4ee191118670 req-44d7b7e8-792e-4b0e-aa43-208a75b1f9c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:14.185350 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-44d7b7e8-792e-4b0e-aa43-208a75b1f9c8 service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.2189732 May 24 10:19:14.230940 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:14.239670 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:19:14.256848 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:14.257959 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:14.323334 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-d90551d2-bf36-42c4-975c-34b02511b328 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 200 len: 217 time: 0.1311216 May 24 10:19:14.465983 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-576367ba-7e43-41c5-810a-4ee191118670 req-edae5b06-9dec-4ac5-af36-e7089805a4a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:14.467536 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-edae5b06-9dec-4ac5-af36-e7089805a4a4 service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.1353316 May 24 10:19:14.503492 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:19:14.550547 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9abfb1f0-35da-4f72-9a15-d0af9053f064 req-98d8f6a4-1470-4f1a-a3fd-b033b75b682c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:14.552106 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9abfb1f0-35da-4f72-9a15-d0af9053f064 req-98d8f6a4-1470-4f1a-a3fd-b033b75b682c tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.2230122 May 24 10:19:14.552833 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:14.618826 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-576367ba-7e43-41c5-810a-4ee191118670 req-693f05bb-281c-4bb9-9152-6f693b6d0f62 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:14.621355 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-693f05bb-281c-4bb9-9152-6f693b6d0f62 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1456263 May 24 10:19:14.683915 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-9abfb1f0-35da-4f72-9a15-d0af9053f064 req-d757774f-b0b1-4cd4-a9a1-e7a08fa3cc93 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1196957 May 24 10:19:14.791576 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-7a0cfca0-505a-4c26-96f1-1463d5cc376b service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1560369 May 24 10:19:14.966717 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-576367ba-7e43-41c5-810a-4ee191118670 req-28143cef-b276-4531-ba02-595f0f2d01fb service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1682482 May 24 10:19:15.053778 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:15.061401 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b5073dee-bed0-4916-9ef9-6ad005def92b None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_floatingips': [{'id': '943c5695-b38a-44fd-9bb5-a0a4de803e89', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'floating_ip_address': '172.27.0.220', 'floating_network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'router_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'port_id': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'fixed_ip_address': '172.20.0.12', 'status': 'ACTIVE', 'standard_attr_id': 684, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'mac_address': 'fa:16:3e:18:73:17', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6f59907d-df21-4f39-8704-0140f911e592', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T10:15:42Z', 'updated_at': '2021-05-24T10:15:45Z', 'revision_number': 1, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:19:15.388361 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f6646b2c-901e-450b-a0e0-14b1f2e1e7a1 None None] New status for floating IP 7df5eb14-1837-465a-8434-005a711bdbcc: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:19:15.968187 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba721e23-9926-49e9-9562-69c7c282718d req-562e8e00-18ef-4aa5-9d4e-8121a26dace9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:15.969090 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ba721e23-9926-49e9-9562-69c7c282718d req-562e8e00-18ef-4aa5-9d4e-8121a26dace9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109112) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:19:16.020344 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-fc8fff2e-9441-478c-8b49-25c7f7e5117a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:16.022047 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-fc8fff2e-9441-478c-8b49-25c7f7e5117a tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.2407432 May 24 10:19:16.043197 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.quota.resource [req-ba721e23-9926-49e9-9562-69c7c282718d req-562e8e00-18ef-4aa5-9d4e-8121a26dace9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109112) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:19:16.044847 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ba721e23-9926-49e9-9562-69c7c282718d req-562e8e00-18ef-4aa5-9d4e-8121a26dace9 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 945 time: 0.2100956 May 24 10:19:16.126416 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-ba721e23-9926-49e9-9562-69c7c282718d req-f960fef8-b037-460e-8459-ef02c9723225 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0742323 May 24 10:19:16.183843 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port f1ec19f0-7419-4db3-9689-1d76c1528de8 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:16.184197 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:16.346347 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] For port f1ec19f0-7419-4db3-9689-1d76c1528de8, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=f1ec19f0-7419-4db3-9689-1d76c1528de8,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:16.346917 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:16.483230 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for f1ec19f0-7419-4db3-9689-1d76c1528de8 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:16.695436 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:16.701633 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-654acda8-d0aa-4a3c-999b-79d8e1477393 tempest-TestShelveInstance-1132599467 tempest-TestShelveInstance-1132599467-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:16.750239 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=f1ec19f0-7419-4db3-9689-1d76c1528de8,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109113) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:16.809279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:16.848200 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] No DVR routers for this DVR port f1ec19f0-7419-4db3-9689-1d76c1528de8 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:16.851419 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:16.854492 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'name': 'network-vif-deleted', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:19:16.907179 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bd96baf8-8c02-4183-a815-a72afc46f394', 'name': 'network-vif-deleted', 'tag': 'f1ec19f0-7419-4db3-9689-1d76c1528de8', 'status': 'completed', 'code': 200} May 24 10:19:16.952496 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-47619994-bb5e-4700-82a9-abc56eddadc6 req-1af51fe5-1457-4933-885a-17932af1cf6f tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "DELETE /v2.0/ports/f1ec19f0-7419-4db3-9689-1d76c1528de8 HTTP/1.1" status: 204 len: 173 time: 0.9245043 May 24 10:19:17.306279 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-c56a3c9d-6162-4dbd-a86a-71bf664c45e9 req-6c82d518-231d-4bac-8ca3-cf223fbef3a2 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.0453699 May 24 10:19:18.510530 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-c33d9758-81c1-4dbf-b711-013fc34fc364 req-c1657392-b946-47a2-a04b-b8075163331d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:19:18.620281 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-c33d9758-81c1-4dbf-b711-013fc34fc364 req-c1657392-b946-47a2-a04b-b8075163331d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:19:18.621513 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-c33d9758-81c1-4dbf-b711-013fc34fc364 req-c1657392-b946-47a2-a04b-b8075163331d tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=bd96baf8-8c02-4183-a815-a72afc46f394 HTTP/1.1" status: 200 len: 210 time: 0.1626227 May 24 10:19:19.177931 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f1ec19f0-7419-4db3-9689-1d76c1528de8 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:19.215896 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: f1ec19f0-7419-4db3-9689-1d76c1528de8 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:19.216241 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device f1ec19f0-7419-4db3-9689-1d76c1528de8 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:20.340426 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:22.471169 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34628) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:22.764486 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port f0501b6b-549c-4b36-8506-eacc6c2f511f {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:22.764829 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:22.910120 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:22.910690 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:23.108657 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for f0501b6b-549c-4b36-8506-eacc6c2f511f owned by network:floatingip {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:23.357771 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:23.361932 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b05df9b5-03d8-4959-b5c3-33ba1ee8f05f tempest-AttachVolumeNegativeTest-100008858 tempest-AttachVolumeNegativeTest-100008858-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:23.410740 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=f0501b6b-549c-4b36-8506-eacc6c2f511f,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:23.442509 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:23.636359 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:23.637373 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.db.l3_db [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Floating IP 943c5695-b38a-44fd-9bb5-a0a4de803e89 disassociated (deleted). External IP: 172.27.0.220, port: 2faafcb9-9d9d-4bf2-997a-509be060bb30. May 24 10:19:24.064947 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109113) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:19:24.260992 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109113) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:19:24.276234 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:19:24.294453 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:24.295528 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:24.347169 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:24.409319 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-7fb5e6c7-6cbd-4410-a383-e3efd96dfd66 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "DELETE /v2.0/floatingips/943c5695-b38a-44fd-9bb5-a0a4de803e89 HTTP/1.1" status: 204 len: 168 time: 1.9366817 May 24 10:19:24.410105 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:19:24.477856 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'status': 'completed', 'code': 200} May 24 10:19:24.521766 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:24.588556 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-a511b1a2-8cc5-42c3-8f88-47c65aafc720 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:24.591117 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-a511b1a2-8cc5-42c3-8f88-47c65aafc720 service neutron] 10.222.0.95 "GET /v2.0/ports?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 1099 time: 0.0971403 May 24 10:19:24.739382 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-80f9e55b-4676-4412-aae7-54bb1c4a0f1b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:24.740866 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-80f9e55b-4676-4412-aae7-54bb1c4a0f1b service neutron] 10.222.0.95 "GET /v2.0/networks?id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 200 len: 912 time: 0.1434829 May 24 10:19:24.814142 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-9d215bf8-80ee-4053-a04f-b0f67feb27c9 service neutron] 10.222.0.95 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 200 len: 217 time: 0.0658805 May 24 10:19:24.888036 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:24.892405 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:19:24.905822 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:24.906449 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:24.910312 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-03381b05-17d8-4012-9be3-ca749783aefb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:24.913287 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-03381b05-17d8-4012-9be3-ca749783aefb service neutron] 10.222.0.95 "GET /v2.0/subnets?id=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 200 len: 851 time: 0.0929613 May 24 10:19:25.025197 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-98d461a5-3137-4b84-9a9b-a48307442ff5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:25.026811 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-98d461a5-3137-4b84-9a9b-a48307442ff5 service neutron] 10.222.0.95 "GET /v2.0/ports?network_id=3670d884-4bc4-4d4e-8dbb-1165ed224ea7&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1061294 May 24 10:19:25.091626 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ec69647-fb0c-4bb9-a1bc-2835083e353d req-b3b9c9cd-d791-4e99-9431-5312d18ccb34 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109112) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:25.094009 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9ec69647-fb0c-4bb9-a1bc-2835083e353d req-b3b9c9cd-d791-4e99-9431-5312d18ccb34 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.1037574 May 24 10:19:25.156730 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [req-9ec69647-fb0c-4bb9-a1bc-2835083e353d req-8f9aa90a-390e-4524-a097-adea1d16e1b1 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0503480 May 24 10:19:25.180969 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:25.215254 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-c91da3e5-f3fe-4b34-a8d4-996b94763020 service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1803868 May 24 10:19:25.374094 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-062966a5-f6b8-46d2-ac0d-8950f69a0db7 req-0e4d5f7b-ecc8-4b43-936a-56429f45dc3f service neutron] 10.222.0.95 "GET /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1484108 May 24 10:19:25.597698 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:25.603122 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bb9a6702-5765-489f-bd09-3386a1f0b700 None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:12:15Z', 'revision_number': 4, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi', '_interfaces': [{'id': 'b64ed3ba-3823-4d02-9656-ce21e0789e74', 'name': '', 'network_id': '3670d884-4bc4-4d4e-8dbb-1165ed224ea7', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'mac_address': 'fa:16:3e:6c:34:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_interface', 'standard_attr_id': 657, 'fixed_ips': [{'subnet_id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:14Z', 'updated_at': '2021-05-24T10:12:23Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'subnets': [{'id': 'f6530c6a-9ae5-4ea6-bf67-c014f1f46f51', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:19:25.831587 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-be951d1d-1ff5-417a-bff2-20c8d89c0de7 None None] New status for floating IP 943c5695-b38a-44fd-9bb5-a0a4de803e89: DOWN {{(pid=109115) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 10:19:26.577906 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27d4e953-2b88-4b60-9055-1d88fc64f401 req-f5c0faf0-779e-482f-9954-05e6ed1a1ffc tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:26.578638 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-27d4e953-2b88-4b60-9055-1d88fc64f401 req-f5c0faf0-779e-482f-9954-05e6ed1a1ffc tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:19:26.596794 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-1f3334b1-1c6f-460c-a6ae-78aac270d2d7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=109113) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 10:19:26.597698 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-1f3334b1-1c6f-460c-a6ae-78aac270d2d7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:19:26.666610 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-27d4e953-2b88-4b60-9055-1d88fc64f401 req-f5c0faf0-779e-482f-9954-05e6ed1a1ffc tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:19:26.667886 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-27d4e953-2b88-4b60-9055-1d88fc64f401 req-f5c0faf0-779e-482f-9954-05e6ed1a1ffc tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.3902755 May 24 10:19:26.668953 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-1f3334b1-1c6f-460c-a6ae-78aac270d2d7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:19:26.670026 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-1f3334b1-1c6f-460c-a6ae-78aac270d2d7 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 946 time: 0.1930330 May 24 10:19:26.776632 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-27d4e953-2b88-4b60-9055-1d88fc64f401 req-ff0f80b9-be2e-43e4-a059-e818b3899185 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/security-groups?id=2b6d1377-7673-408e-86ee-8125745d3311&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1029701 May 24 10:19:26.853651 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port 2faafcb9-9d9d-4bf2-997a-509be060bb30 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:26.853993 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:27.001866 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.db [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] For port 2faafcb9-9d9d-4bf2-997a-509be060bb30, host hv-cinder-79123, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-79123',level=0,port_id=2faafcb9-9d9d-4bf2-997a-509be060bb30,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109113) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:27.002236 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:27.141968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for 2faafcb9-9d9d-4bf2-997a-509be060bb30 owned by compute:nova {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:27.329035 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:27.335648 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-b8f5390f-97d5-4001-8758-8ae5beadee8a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:27.386968 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=2faafcb9-9d9d-4bf2-997a-509be060bb30,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:27.390410 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:27.467181 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.db.l3_dvrscheduler_db [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] No DVR routers for this DVR port 2faafcb9-9d9d-4bf2-997a-509be060bb30 on host hv-cinder-79123 {{(pid=109113) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:27.469939 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for port, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:27.472856 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'name': 'network-vif-deleted', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30'}] {{(pid=109113) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 10:19:27.523652 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '6f59907d-df21-4f39-8704-0140f911e592', 'name': 'network-vif-deleted', 'tag': '2faafcb9-9d9d-4bf2-997a-509be060bb30', 'status': 'completed', 'code': 200} May 24 10:19:27.563959 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-0f26dea0-1814-42eb-8e46-bd1e916c2630 req-cadeb098-f7ed-40f9-82a6-7ffcde24aa4e tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.95 "DELETE /v2.0/ports/2faafcb9-9d9d-4bf2-997a-509be060bb30 HTTP/1.1" status: 204 len: 173 time: 0.8763330 May 24 10:19:27.981495 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d2a9cf43-0c72-4dd4-8cd8-190712a91a9b req-6049f1d7-7c0d-47b7-a718-b224cdcef761 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Synchronizing usage tracker for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 10:19:28.117984 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.quota.resource [req-d2a9cf43-0c72-4dd4-8cd8-190712a91a9b req-6049f1d7-7c0d-47b7-a718-b224cdcef761 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Unset dirty status for tenant:2cc9bd0e4be64b0e9add9a9f9e8bbe52 on resource:port {{(pid=109113) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 10:19:28.119188 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [req-d2a9cf43-0c72-4dd4-8cd8-190712a91a9b req-6049f1d7-7c0d-47b7-a718-b224cdcef761 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "GET /v2.0/ports?device_id=6f59907d-df21-4f39-8704-0140f911e592 HTTP/1.1" status: 200 len: 210 time: 0.1859610 May 24 10:19:29.209841 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 2faafcb9-9d9d-4bf2-997a-509be060bb30 no longer exists at agent hyperv_hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:29.220935 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34646) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:29.250309 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] No Port match for: 2faafcb9-9d9d-4bf2-997a-509be060bb30 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:29.250715 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Device 2faafcb9-9d9d-4bf2-997a-509be060bb30 not bound to the agent host hv-cinder-79123 {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:29.458044 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Deleting port a5af3e6d-7429-4272-bdf8-b962bdea1b93 {{(pid=109113) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:29.458549 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:29.588311 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:29.588640 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:29.736284 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.plugins.ml2.plugin [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Calling delete_port for a5af3e6d-7429-4272-bdf8-b962bdea1b93 owned by network:floatingip {{(pid=109113) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:29.940430 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-129003717'] for port, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:29.945135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-bde80f29-1855-45c6-88f4-bb1a35f91e42 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:29.996135 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'Port': ['ID=a5af3e6d-7429-4272-bdf8-b962bdea1b93,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:29.999272 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:30.099086 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for floatingip, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:30.174605 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-4104059'] for floatingip, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:30.175770 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-1a49ae73-6e51-42bd-89f5-a35ccef665fd tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "DELETE /v2.0/floatingips/6ed2d069-1420-4918-b860-85d659542cc3 HTTP/1.1" status: 204 len: 168 time: 0.9538033 May 24 10:19:31.750584 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.wsgi [-] (109113) accepted ('10.222.0.67', 34660) {{(pid=109113) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:31.846661 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, before_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:31.884279 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, precommit_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:32.048889 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:32.053159 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-1d15e4c9-9e97-4f12-9918-607511468e25 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:32.070386 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=2b6d1377-7673-408e-86ee-8125745d3311,revision_number=None']} {{(pid=109113) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:32.073084 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG oslo_concurrency.lockutils [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=109113) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:32.180192 dv-cinder-791239-2-iscsi neutron-server[109113]: DEBUG neutron_lib.callbacks.manager [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] Notify callbacks [] for security_group, before_response {{(pid=109113) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:32.181409 dv-cinder-791239-2-iscsi neutron-server[109113]: INFO neutron.wsgi [None req-be2a9f4b-4603-4187-aa6f-bbd667033932 tempest-TestVolumeBootPattern-2018797883 tempest-TestVolumeBootPattern-2018797883-project] 10.222.0.67 "DELETE /v2.0/security-groups/2b6d1377-7673-408e-86ee-8125745d3311 HTTP/1.1" status: 204 len: 168 time: 0.4294381 May 24 10:19:32.309454 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34666) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:33.203801 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:33.334371 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-25e836d9-f015-48b8-a9ed-c857b41b09df None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:33.667380 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Deleting port b64ed3ba-3823-4d02-9656-ce21e0789e74 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:33.668032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:33.799462 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] For port b64ed3ba-3823-4d02-9656-ce21e0789e74, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=b64ed3ba-3823-4d02-9656-ce21e0789e74,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:33.799738 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:33.941620 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Calling delete_port for b64ed3ba-3823-4d02-9656-ce21e0789e74 owned by network:router_interface {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:34.351923 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:34.356309 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:34.401568 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Pushing event deleted for resources: {'Port': ['ID=b64ed3ba-3823-4d02-9656-ce21e0789e74,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:34.404998 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:34.888220 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:35.280121 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Router 6db26429-7a8b-455b-8586-398e35a18a57 has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:19:35.510192 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:19:35.520600 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Sync routers for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:19:35.524214 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] 10.222.0.67 "PUT /v2.0/routers/6db26429-7a8b-455b-8586-398e35a18a57/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2135215 May 24 10:19:35.528533 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34672) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:35.535167 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:35.535946 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:06Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:35.737517 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: dv-cinder-791239-2-iscsi {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:35.789034 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks [] for router, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:35.977598 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.138590 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Deleting port 9a20f529-25b1-4304-a42c-29516af43de2 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:36.138859 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.156965 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9cf830f6-8092-4891-ae00-84ac47f79838 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.159427 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.276247 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] For port 9a20f529-25b1-4304-a42c-29516af43de2, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=9a20f529-25b1-4304-a42c-29516af43de2,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:36.276608 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.339758 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138'}], 'enable_snat': True}, 'gw_port_id': '9a20f529-25b1-4304-a42c-29516af43de2', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:19:34Z', 'revision_number': 5, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port': {'id': '9a20f529-25b1-4304-a42c-29516af43de2', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:0b:05:51', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6db26429-7a8b-455b-8586-398e35a18a57', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:10Z', 'updated_at': '2021-05-24T10:12:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:36.346480 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Checking router: 6db26429-7a8b-455b-8586-398e35a18a57 for host: dv-cinder-791239-2-iscsi {{(pid=109114) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:19:36.357730 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Routers {'6db26429-7a8b-455b-8586-398e35a18a57'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109114) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:36.358338 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['6db26429-7a8b-455b-8586-398e35a18a57'], 'active': True} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:36.404174 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Calling delete_port for 9a20f529-25b1-4304-a42c-29516af43de2 owned by network:router_gateway {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:36.526114 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvr_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] Set router 6db26429-7a8b-455b-8586-398e35a18a57 gateway port host: None {{(pid=109114) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:36.644707 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_hamode_db [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:19:35Z', 'revision_number': 6, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port_host': None}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109114) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:36.649860 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4bb5ddd-970f-4eac-b0fa-9e9a58553f08 None None] The sync data for ids ['6db26429-7a8b-455b-8586-398e35a18a57'] in dv-cinder-791239-2-iscsi is: [{'id': '6db26429-7a8b-455b-8586-398e35a18a57', 'name': 'tempest-TestVolumeBootPattern-1417893617-router', 'tenant_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:09Z', 'updated_at': '2021-05-24T10:19:35Z', 'revision_number': 6, 'project_id': '2cc9bd0e4be64b0e9add9a9f9e8bbe52', 'gw_port_host': None}] {{(pid=109114) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:19:36.780428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:36.785490 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-62bb9ed4-0bcb-4e2d-baa9-152dee2e00d2 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:36.858868 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Pushing event deleted for resources: {'Port': ['ID=9a20f529-25b1-4304-a42c-29516af43de2,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:36.861635 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:36.934434 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.104838 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b64ed3ba-3823-4d02-9656-ce21e0789e74 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:37.138328 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.140827 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.142702 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: b64ed3ba-3823-4d02-9656-ce21e0789e74 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:37.143055 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device b64ed3ba-3823-4d02-9656-ce21e0789e74 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:37.248554 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.249130 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 6db26429-7a8b-455b-8586-398e35a18a57 {{(pid=109112) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:19:37.265263 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.266388 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-b8123257-64ef-461e-aeb7-4a1364053a61 admin admin] 10.222.0.67 "DELETE /v2.0/routers/6db26429-7a8b-455b-8586-398e35a18a57 HTTP/1.1" status: 204 len: 168 time: 1.7367706 May 24 10:19:37.269225 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34676) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:37.277271 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.335867 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Deleting subnet f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:19:37.339427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.396478 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.493858 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.900638 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.904887 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ee1896a7-a09c-4225-8772-7195cd1af9d5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:37.941617 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=f6530c6a-9ae5-4ea6-bf67-c014f1f46f51,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:37.943625 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:37.946404 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:37.950053 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0228b520-fe66-458f-82a5-e4dc87822638 None None] Network 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:19:37.952213 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-ee867e5d-cce6-4946-9a31-89b9c241be60 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/f6530c6a-9ae5-4ea6-bf67-c014f1f46f51 HTTP/1.1" status: 204 len: 168 time: 0.6814673 May 24 10:19:37.954858 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34680) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:38.071608 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.165427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Deleting port bb5b081c-fde7-4536-9096-eefa2bf2806a {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:38.166254 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.250828 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] DHCP port deletion for 3670d884-4bc4-4d4e-8dbb-1165ed224ea7 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:19:38.308427 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] For port bb5b081c-fde7-4536-9096-eefa2bf2806a, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=bb5b081c-fde7-4536-9096-eefa2bf2806a,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:38.308804 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.332271 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Deleting port bb5b081c-fde7-4536-9096-eefa2bf2806a {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:38.332645 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.454107 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Calling delete_port for bb5b081c-fde7-4536-9096-eefa2bf2806a owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:38.510418 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] For port bb5b081c-fde7-4536-9096-eefa2bf2806a, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=bb5b081c-fde7-4536-9096-eefa2bf2806a,segment=NetworkSegment(190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99),segment_id=190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:38.510867 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.591263 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.597496 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-0757b0d1-3a59-42e2-a072-663312ae0278 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:38.643489 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Pushing event deleted for resources: {'Port': ['ID=bb5b081c-fde7-4536-9096-eefa2bf2806a,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:38.646355 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:38.647080 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Calling delete_port for bb5b081c-fde7-4536-9096-eefa2bf2806a owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:38.650704 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] No DVR routers for this DVR port bb5b081c-fde7-4536-9096-eefa2bf2806a on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:38.719507 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.724554 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:38.791816 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Pushing event deleted for resources: {'Port': ['ID=bb5b081c-fde7-4536-9096-eefa2bf2806a,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:38.795189 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] No DVR routers for this DVR port bb5b081c-fde7-4536-9096-eefa2bf2806a on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:38.804634 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.804815 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.805223 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.805480 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.805810 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.806549 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.806987 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.807378 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.807709 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.808032 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.808354 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.808670 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-791239-2-iscsi {{(pid=109114) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 10:19:38.815007 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-75062fdf-5a3a-4d53-a7f2-8c921f3a93da None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:38.883351 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.896065 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.911731 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['3670d884-4bc4-4d4e-8dbb-1165ed224ea7']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:38.929580 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:38.930049 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '190d52fc-7be5-48bd-aeb8-f1e6ecbc0f99') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:38.944759 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.974347 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.978155 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Releasing vlan 82 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:19:38.978596 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:19:38.993080 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:19:39.012527 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 9a20f529-25b1-4304-a42c-29516af43de2 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:39.050121 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: 9a20f529-25b1-4304-a42c-29516af43de2 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:39.050456 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device 9a20f529-25b1-4304-a42c-29516af43de2 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:39.050883 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device bb5b081c-fde7-4536-9096-eefa2bf2806a no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:39.091246 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: bb5b081c-fde7-4536-9096-eefa2bf2806a {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:39.091513 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device bb5b081c-fde7-4536-9096-eefa2bf2806a not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:39.156973 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:39.161389 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-3b4a5fa8-fb67-4a9b-9590-b7d25dae37c7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:39.176820 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:39.188151 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Pushing event deleted for resources: {'Network': ['ID=3670d884-4bc4-4d4e-8dbb-1165ed224ea7,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:39.191753 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:39.261982 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-cd3572b3-57b5-4d50-93e2-fdf7aa067d5b admin admin] 10.222.0.67 "DELETE /v2.0/networks/3670d884-4bc4-4d4e-8dbb-1165ed224ea7 HTTP/1.1" status: 204 len: 168 time: 1.3057492 May 24 10:19:39.264733 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34684) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:39.942525 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-425072'] for router_interface, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:40.469456 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Deleting port d3ce81dc-f3b2-4725-90b6-77a7e878de97 {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:40.470280 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:40.603053 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] For port d3ce81dc-f3b2-4725-90b6-77a7e878de97, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=d3ce81dc-f3b2-4725-90b6-77a7e878de97,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:40.603476 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:40.836299 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Calling delete_port for d3ce81dc-f3b2-4725-90b6-77a7e878de97 owned by network:router_interface {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:41.083050 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:41.088626 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-a01ddde2-c43c-4b9a-a9d3-593f64fed203 tempest-TestMinimumBasicScenario-1222037637 tempest-TestMinimumBasicScenario-1222037637-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:41.134881 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Pushing event deleted for resources: {'Port': ['ID=d3ce81dc-f3b2-4725-90b6-77a7e878de97,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:41.138376 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:41.476556 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-3931038', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-424160'] for router_interface, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:41.877191 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad has already been hosted by L3 agent f0e0c719-319c-4f00-8ca3-495f176487fd {{(pid=109112) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 10:19:42.069894 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] Notify agent at l3_agent.dv-cinder-791239-2-iscsi the message routers_updated {{(pid=109112) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 10:19:42.080744 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Sync routers for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 10:19:42.093319 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:42.094069 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:42.164808 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-61cfa947-af55-4137-b68d-38feb14c51d4 admin admin] 10.222.0.67 "PUT /v2.0/routers/8f6d9311-60c1-4a8b-abc6-ac26dcd1daad/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.8987455 May 24 10:19:42.169520 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34688) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:42.312439 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:42.373496 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks [] for router, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:42.717897 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:42.728103 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:19:40Z', 'revision_number': 5, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:42.732178 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Checking router: 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad for host: dv-cinder-791239-2-iscsi {{(pid=109115) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 10:19:42.740604 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvrscheduler_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Routers {'8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'} bound to L3 agent in host dv-cinder-791239-2-iscsi {{(pid=109115) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 10:19:42.741813 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-791239-2-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T09:52:06Z,description=,heartbeat_timestamp=2021-05-24T10:19:36Z,host='dv-cinder-791239-2-iscsi',id=f0e0c719-319c-4f00-8ca3-495f176487fd,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T09:52:06Z,topic='l3_agent')) {'router_ids': ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'], 'active': True} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:42.903972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Deleting port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:42.904251 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:42.994481 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_dvr_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] Set router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad gateway port host: dv-cinder-791239-2-iscsi {{(pid=109115) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 10:19:43.062877 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] For port eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,segment=NetworkSegment(7bdf7a39-c927-428c-ae01-d27b28cde973),segment_id=7bdf7a39-c927-428c-ae01-d27b28cde973)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:43.063437 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:43.269663 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Calling delete_port for eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd owned by network:router_gateway {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:43.398206 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.l3_hamode_db [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:19:40Z', 'revision_number': 5, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}], 'dv-cinder-791239-2-iscsi', False) {} {{(pid=109115) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:43.403608 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-45cf8ab9-30f2-4b61-a466-0eb0e76e498f None None] The sync data for ids ['8f6d9311-60c1-4a8b-abc6-ac26dcd1daad'] in dv-cinder-791239-2-iscsi is: [{'id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'name': 'tempest-TestVolumeBootPattern-949955037-router', 'tenant_id': '0850b91daf724bcd93114c2b6acaf926', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'external_fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125'}], 'enable_snat': True}, 'gw_port_id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T10:12:20Z', 'updated_at': '2021-05-24T10:19:40Z', 'revision_number': 5, 'project_id': '0850b91daf724bcd93114c2b6acaf926', 'gw_port': {'id': 'eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd', 'name': '', 'network_id': '2211f181-5fdc-4580-9f8a-8610ff86d56a', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:6b:5d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '8f6d9311-60c1-4a8b-abc6-ac26dcd1daad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 668, 'fixed_ips': [{'subnet_id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'ip_address': '172.27.0.125', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-791239-2-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T10:12:22Z', 'updated_at': '2021-05-24T10:12:29Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'c29e3891-6552-402c-9c30-4a0d6fd7e6a6', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-791239-2-iscsi'}] {{(pid=109115) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 10:19:43.562518 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:43.566653 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-229d2796-179c-4b78-acae-a352fce9b60d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:43.610428 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Pushing event deleted for resources: {'Port': ['ID=eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:43.614335 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:43.721923 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-425325'] for router_gateway, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:43.920463 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-423392', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4221'] for router, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:43.923318 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.082424 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-423900'] for router, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.082865 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 8f6d9311-60c1-4a8b-abc6-ac26dcd1daad {{(pid=109112) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 10:19:44.102214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] Notify callbacks [] for router, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.103645 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-22b542b0-cd56-4369-a4c0-6160b803d4f1 admin admin] 10.222.0.67 "DELETE /v2.0/routers/8f6d9311-60c1-4a8b-abc6-ac26dcd1daad HTTP/1.1" status: 204 len: 168 time: 1.9329438 May 24 10:19:44.107977 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34692) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:44.191636 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.db_base_plugin_v2 [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Deleting subnet 8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb {{(pid=109112) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 10:19:44.195741 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.257317 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-128638743'] for subnet, precommit_delete_associations {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.342800 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-3610836'] for subnet, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.740321 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-3629921', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087544', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-127969809', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1461655'] for subnet, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.746373 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:44.783032 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:44.786210 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:44.787270 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for subnet, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:44.795439 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-7f38b87b-6da5-4a3a-b7d7-5144ccf4a8b0 admin admin] 10.222.0.67 "DELETE /v2.0/subnets/8c2fa100-895f-4fd4-a4b7-986ec0d3a5bb HTTP/1.1" status: 204 len: 168 time: 0.6860437 May 24 10:19:44.796456 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-69fc136b-aff3-4b41-b719-8449cc65fe4d None None] Network 87e2b5b7-b682-411b-ad8f-70bb178250f4 requested from dv-cinder-791239-2-iscsi {{(pid=109114) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 10:19:44.799371 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34696) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:45.026511 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-3553115'] for network, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.028764 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:45.077635 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:45.077968 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device eeb6827f-fffb-4fa2-8eb7-e2e1dde861dd not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:45.078400 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d3ce81dc-f3b2-4725-90b6-77a7e878de97 no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:45.108715 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Deleting port f97a5e38-df31-42ff-b538-474375dde42d {{(pid=109112) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:45.109086 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.116488 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: d3ce81dc-f3b2-4725-90b6-77a7e878de97 {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:45.116822 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device d3ce81dc-f3b2-4725-90b6-77a7e878de97 not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:45.198397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] DHCP port deletion for 87e2b5b7-b682-411b-ad8f-70bb178250f4 request from dv-cinder-791239-2-iscsi {{(pid=109114) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 10:19:45.268101 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Deleting port f97a5e38-df31-42ff-b538-474375dde42d {{(pid=109114) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 10:19:45.268541 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8762070891843'] for port, before_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.281645 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.db [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] For port f97a5e38-df31-42ff-b538-474375dde42d, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=f97a5e38-df31-42ff-b538-474375dde42d,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109112) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:45.282228 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.384645 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.db [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] For port f97a5e38-df31-42ff-b538-474375dde42d, host dv-cinder-791239-2-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-791239-2-iscsi',level=0,port_id=f97a5e38-df31-42ff-b538-474375dde42d,segment=NetworkSegment(9726e60b-1b4d-46d9-9951-468c3217b9eb),segment_id=9726e60b-1b4d-46d9-9951-468c3217b9eb)] {{(pid=109114) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 10:19:45.385075 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-418465'] for port, precommit_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.452631 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.plugin [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Calling delete_port for f97a5e38-df31-42ff-b538-474375dde42d owned by network:dhcp {{(pid=109112) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:45.525684 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Calling delete_port for f97a5e38-df31-42ff-b538-474375dde42d owned by network:dhcp {{(pid=109114) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 10:19:45.585976 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-127970274'] for port, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.590756 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5a741732-85e1-4d7e-804a-fda50faec475 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:45.593397 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-3569359', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-4087549', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-3931020', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8762070893478', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8762070492389'] for port, after_delete {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.599451 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:45.646354 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Pushing event deleted for resources: {'Port': ['ID=f97a5e38-df31-42ff-b538-474375dde42d,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:45.651214 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:45.652448 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] No DVR routers for this DVR port f97a5e38-df31-42ff-b538-474375dde42d on host dv-cinder-791239-2-iscsi {{(pid=109112) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:45.658750 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Pushing event deleted for resources: {'Port': ['ID=f97a5e38-df31-42ff-b538-474375dde42d,revision_number=None']} {{(pid=109114) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:45.661212 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] No DVR routers for this DVR port f97a5e38-df31-42ff-b538-474375dde42d on host dv-cinder-791239-2-iscsi {{(pid=109114) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 10:19:45.664053 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG oslo_concurrency.lockutils [None req-8dd5b1c8-dd04-48d8-bc5c-607961749478 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=109114) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:45.795932 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-3629403', 'neutron.services.segments.db._delete_segments_for_network-8762070920165'] for network, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.810966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['87e2b5b7-b682-411b-ad8f-70bb178250f4']}} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:45.825886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '9726e60b-1b4d-46d9-9951-468c3217b9eb') {'for_net_delete': True} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:45.826199 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.services.segments.db [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '9726e60b-1b4d-46d9-9951-468c3217b9eb') {} {{(pid=109112) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 10:19:45.839287 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8762070928294', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-128638748'] for segment, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.857202 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059'] for segment, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.864357 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Releasing vlan 89 on physical network physnet1 to pool {{(pid=109112) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 10:19:45.869698 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-3609059', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-127970231'] for segment, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: Traceback (most recent call last): May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: timer() May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: cb(*args, **kw) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: return func(*args, **kwargs) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: self._notify() May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: self.callback(batched_events) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: event.method(event) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: aggregate_uuid = self.p_client.list_aggregates( May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: return f(self, *a, **k) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._get(url).json() May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: return self.request(url, 'GET', **kwargs) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: base_url = self.get_endpoint(auth, allow=allow, May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 10:19:45.878705 dv-cinder-791239-2-iscsi neutron-server[109112]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 10:19:46.031384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-3629394', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-3753199', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-3931031', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-425328'] for network, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.035360 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-5209c417-0fd0-4b59-989f-0c9c0699a5a4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:46.049886 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-3931058'] for network, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.062072 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Pushing event deleted for resources: {'Network': ['ID=87e2b5b7-b682-411b-ad8f-70bb178250f4,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:46.068553 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:46.072787 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-c6de7789-ae4b-4f10-908d-b6f77318b0f2 admin admin] 10.222.0.67 "DELETE /v2.0/networks/87e2b5b7-b682-411b-ad8f-70bb178250f4 HTTP/1.1" status: 204 len: 168 time: 1.2723441 May 24 10:19:46.412255 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34702) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:46.467180 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-a9208663-b492-4ae2-86d5-102c9c5033e3 admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=2cc9bd0e4be64b0e9add9a9f9e8bbe52&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0528097 May 24 10:19:46.470059 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34706) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:46.541676 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.570501 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.693943 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.698061 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-d9041a2e-3803-4fad-8911-0832dc99e08f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:46.712633 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1a5958ce-3459-45c8-8025-29ec96e20aae,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:46.716216 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:46.724290 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:46.726209 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-82e888ba-0287-4c34-a9b8-4586b5f39bcc admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/1a5958ce-3459-45c8-8025-29ec96e20aae HTTP/1.1" status: 204 len: 168 time: 0.2544799 May 24 10:19:46.997288 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device f97a5e38-df31-42ff-b538-474375dde42d no longer exists at agent ovs-agent-dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 10:19:47.035152 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.plugin [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] No Port match for: f97a5e38-df31-42ff-b538-474375dde42d {{(pid=109114) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 10:19:47.035505 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron.plugins.ml2.rpc [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Device f97a5e38-df31-42ff-b538-474375dde42d not bound to the agent host dv-cinder-791239-2-iscsi {{(pid=109114) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 10:19:47.502384 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34714) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:47.576647 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-51550257-5d74-4e50-8b09-f839b67ce43e admin admin] 10.222.0.67 "GET /v2.0/security-groups?tenant_id=0850b91daf724bcd93114c2b6acaf926&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0725381 May 24 10:19:47.579927 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [-] (109112) accepted ('10.222.0.67', 34718) {{(pid=109112) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 10:19:47.649728 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:47.686693 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:47.842826 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-3753213'] for security_group, after_delete {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:47.848288 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-edc7ba97-f705-47ba-9cc6-60597eb17592 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 10:19:47.862972 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=53727b75-01be-405f-a6ce-369272629f50,revision_number=None']} {{(pid=109112) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 10:19:47.866624 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG oslo_concurrency.lockutils [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=109112) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 10:19:47.868198 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron_lib.callbacks.manager [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] Notify callbacks [] for security_group, before_response {{(pid=109112) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:47.869570 dv-cinder-791239-2-iscsi neutron-server[109112]: INFO neutron.wsgi [None req-285fcfc9-187b-4aa0-b670-d718a7ec9459 admin admin] 10.222.0.67 "DELETE /v2.0/security-groups/53727b75-01be-405f-a6ce-369272629f50 HTTP/1.1" status: 204 len: 168 time: 0.2879369 May 24 10:19:50.362805 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:19:51.492933 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:20:03.362787 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-47025bed-aea4-403b-88bd-def3a8b0322e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:05.882655 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-09e555ba-8b44-49a7-b09d-dc791d553774 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:05.979013 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:07.281913 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:08.830379 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:20.351980 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:28.492887 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:20:29.218281 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-153a827d-c715-4a22-bd42-c288bea234da req-a3489e7a-aae1-4c34-ab1b-f3da18a9bdc2 service neutron] (109112) timed out ('10.222.0.95', 58147) {{(pid=109112) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 10:20:33.349749 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b8f00133-7435-4efb-8bcc-b762d3bef21c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:35.898684 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-77f833a7-d84f-4829-9dbe-11c96af0f8fd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:35.944960 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:37.304504 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:38.841185 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:20:50.395091 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:03.326328 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-66bddafa-f24c-450e-add8-e6b150cbf90f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:05.495916 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:21:05.881192 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-4b6df667-39de-44bc-9e54-bc8f91eec86a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:05.914713 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:07.291985 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:08.840293 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:20.360641 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:33.377147 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a6197181-f2cf-4fa5-b458-c87bc6e0b1a8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:35.857132 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3e7f703c-e46d-49a1-abc3-48f437f26cb8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:35.943639 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:37.274394 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:38.823037 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:21:42.506376 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:21:50.347834 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:03.434226 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-9b14d39c-3cee-4bb4-893e-bc89a52fe4c7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:05.947790 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-b4ee4f1b-0fe7-4b1c-b233-c7ef1669b8dd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:05.959189 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:07.319592 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:08.867707 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:19.496313 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:22:20.353657 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:33.318570 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-cf01bb79-3237-44fb-ad89-ce925b3c20dd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:35.936137 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-0ceccc2f-83f7-4823-be21-e96b413c30c6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:35.970180 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:37.312993 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:38.884306 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:50.358560 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:22:56.495294 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:23:03.335617 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-0402ab68-fb34-4bab-afc3-a40c3db2403a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:05.855221 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-a97b0077-923f-48d8-a683-e0b834ab943d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:05.997875 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:07.306377 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:08.854896 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:20.420370 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:33.340533 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-68e99970-fe1a-4765-b10a-c978720119a8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:33.495181 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:23:35.848550 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-3fc5403b-d6ee-4c2b-8e7a-62c9f871f055 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:35.939953 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:37.436669 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:38.877751 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:23:50.477112 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:03.316607 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-88d22505-be2a-4c86-8f2d-c26e980417ed None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:05.834246 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-ca541e82-ab56-4ef5-93fd-92bbff84e277 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:05.905248 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:07.385148 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:08.865331 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:10.494871 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:24:20.402303 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:33.362046 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-c0a8072c-a1d3-4fc3-81a5-fef9749b5158 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:35.848097 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-035a3488-1f09-4ccc-8a42-025b9c093ec6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:35.944738 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-2c794f37-d027-483f-a4e6-a3a26bac64b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:37.333255 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-fc6a4d54-2939-4767-9a33-9cea3376aa5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:38.898832 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-85ae6e31-9361-4561-9e80-a4292675e4e6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:47.494108 dv-cinder-791239-2-iscsi neutron-server[109115]: DEBUG neutron.db.agents_db [None req-aa497d64-b72a-434b-8398-ecdcbc1bfff0 None None] Agent healthcheck: found 6 active agents {{(pid=109115) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 10:24:49.629557 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-e1e19eff-2321-4a16-858a-b884163879bc req-a1b58eba-8159-4825-b36a-6c8dc40025a3 service neutron] (109112) timed out ('10.222.0.95', 58118) {{(pid=109112) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 10:24:50.424078 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-e046ca3e-b9a0-41a9-b689-073b329e1ffd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 10:24:54.277966 dv-cinder-791239-2-iscsi neutron-server[109112]: DEBUG neutron.wsgi [req-93ca72b1-7fde-4bbb-b6ca-bd429d322645 req-903f550b-a5f9-4713-a207-7a977773eb10 service neutron] (109112) timed out ('10.222.0.67', 59320) {{(pid=109112) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 10:25:03.351364 dv-cinder-791239-2-iscsi neutron-server[109114]: DEBUG neutron_lib.callbacks.manager [None req-f7e4f9c2-a2c2-427d-984f-91aa7142d660 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8762070920183', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-3631253'] for agent, after_update {{(pid=109114) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}}